ABSTRACT
01.Two verbs
Generative AI is a fine thing for making new things: a prompt goes in, new content comes out, and the point is that the result is new. Translation has the opposite aim. A performance goes in, and the same performance comes out in another language. Any translator, human or machine, is judged on how little it invents.
| Generate | Translate | |
|---|---|---|
| In | A prompt | A finished performance |
| Out | New content | The same performance in another language |
| Measured by | How new and good the result is | How faithful it stays to the original |
02.What stays, what changes
| Stays | Changes |
|---|---|
| The actor | The words |
| The face | The mouth movements |
| The timing | The voice track, in the new language |
| The scene sound: laughter, music, ambience |
The voice in the new language is the actor's own, taken from the clip itself. Names and catchphrases stay exactly as said, through a Do Not Translate list; jokes are rewritten to land in the target language.
03.Why lip-sync is part of translation
People do not hear speech with their ears alone. In the McGurk effect, a voice saying [b] dubbed onto a face saying [g] is usually heard as [d], a sound neither channel carried (Tiippana, 2014, describing McGurk and MacDonald, 1976). The brain fuses what it sees with what it hears.
A voice-only dub leaves that mismatch in every frame: the mouth says one language and the soundtrack another, so comprehension gets harder. That makes the mouth part of the message. Translating a performance means translating the lips with the words, which is why Familiar renders voice and lip-sync together (the FAQ, "Why does lip-sync matter?").
04.The honest nuance
For machine learning readers: the model does render pixels and audio. It redraws the mouth region for the new line and speaks the new line in the actor's voice. That is why the longer form reads "not generative AI in the traditional sense." The difference is the input. Nothing is created from a prompt and nothing new is made up; every frame and every sound answers to a line that was already performed.
05.Creative directors review before it goes out
Some lines can land more than one way: a joke, a jab, a line that reads sincere or dry. Those come back as three translations with different intent and emotion. The team picks one, edits any line against the original, and only then triggers the render. Teams that trust the flow can skip the stop.
06.Machine learning for the tedious part.
The tedious part of dubbing is the volume: every line, in every language, voiced and synced to the mouth. Machine learning carries that. The choices that make a dub good, which joke lands, which tone fits, which name stays, remain with the people who made the work.
All of the data we trained with ourselves are actually generated by the Familiar team.
We're another tool for the creative process.
QUESTIONS, ANSWERED
Is Familiar generative AI?
Not in the traditional sense. It is a multimodal translation tool: no prompts, nothing new made up. The model renders the mouth and the voice for each translated line of a performance that already exists.
What happens to my music and sound effects?
They stay. The laughter, the music and bass, the sound effects and the ambience survive under the new dialogue.
Can I fix the translation?
Yes. The pipeline pauses at a review screen where you edit any line against the original before it renders.
REFERENCES
Multimodal Translation
They're a team of 8 with PhDs, backed by Samsung, RedBird, Gold House. Our mission
FAMILIAR · THE LAUNCH FILM · 2:31