DEFINITION

What Is a Multimodal Localization Foundation Model (MLFM)?

FAMILIAR LABS RESEARCHUPDATED ALL ARTICLES
READ THIS INEnglishEspañolPortuguês中文日本語한국어FrançaisDeutschالعربيةBahasa IndonesiaРусскийItalianoTiếng ViệtไทยNederlandsΕλληνικάУкраїнськаHrvatskiLietuviųSlovenčinaSvenskaБългарскиDanskLatviešu

ABSTRACT

The first foundation model for multimodal translation, a new paradigm, the first of its kind. MLFM is one model that decides what to localize and how, across text, audio and visual, and shows you the choices that matter. Translation first, then the dub in one shot: the voice, the lips and the scene together. The decisions agree because one model makes them. The person never changes.
THE FIRST FOUNDATION MODEL FOR MULTIMODAL TRANSLATIONAUDIO-VISUAL TRANSLATION AT ONCE, NOT CASCADED

What does MLFM decide?

TEXT
Which rendering carries a line into the new language. The MLFM weighs several renderings and picks the best: in a joke, a story or a conversational turn, one word changes the intention and the emotion. The same decisions cover the terms, what not to translate, the slang kept as said, the name and the brand kept, a reference swapped for one the audience knows, what the culture avoids, and the profanity lowered for a younger cut.
AUDIO
What should be translated at all. The MLFM hears everything in the track: the laughter, the crowd, a shout, a speaker in the background no one can make out. It finds the main voice, carries it into the new language in the speaker's own voice, and keeps the rest as it was.
VISUAL
Who is actually saying it. Audio and visual are decided together, so the MLFM knows who narrates, who speaks from off screen, and whose lips should move. The lips follow the new words in every line, occlusions and spatial reasoning included; the expressions and gestures follow when the emphasis or the culture calls for it.

THE CHOICES YOU SEE

TEXT
TODAYWhere a line can land more than one way, your creative directors get the renderings with the reasoning behind each and pick one. A podcast, an episode or a movie can carry many such segments.
IN THE WORKS
AUDIO
TODAY
IN THE WORKSChoosing what stays in the original language. A chant, a yell or a phrase can stay as said when you want it to.
VISUAL
TODAY
IN THE WORKSPicking a different face when the model chose the wrong speaker, and choosing which text in the frame to translate. That text, a phone screen, a book cover, a street sign, a store front, will be written in the terms you already chose; it decides on the fly what stays as shot, a brand, a name, slang: a bag of Tostitos stays Tostitos.

Why one model?

A CHAIN OF TOOLS

  • Speech-to-text, then translation, then a voice, then lip-sync, each stage tuned on its own.
  • That chain made machine dubbing possible.
  • Each stage still guesses without the others, so the errors compound from one stage to the next.

ONE FOUNDATION MODEL, THE FIRST OF ITS KIND

  • MLFM makes the decisions together.
  • The translation knows the mouth it has to fit.
  • The voice knows the scene it sits in.
  • The person never changes.
++
Translation first, then the dub in one shot: the voice, the lips and the scene come out together, from the same decisions.

Audio-visual translation at once, not cascaded: the first of its kind. That is what makes MLFM the first foundation model for multimodal translation: one model that carries the words, the voice, the lips and the scene into a new language, all at once. The high-quality route today is assistive: a team of VFX artists and weeks per project, every language a new project. The one-shot tools today are fast and give up the rest. Familiar Labs is one shot and the quality.

HeyGenHeyGenElevenLabsElevenLabsHeyGen + ElevenLabsONE-SHOT TOOLS · NO CONTROL

One shot today means giving up the rest.

The one-shot tools today give up the rest: the translation cannot be edited, the music and the room drop out, the voice drifts, and the lips fail on a hand over the face, a microphone, blur, bokeh, a side view or several people in frame. No spatial reasoning, no understanding of the world, no control. ElevenLabs' current Dubbing v2 edits only through an Enterprise API, and its full editor is the older model, in maintenance mode; HeyGen stitches a lip-sync over a dub like that, and the lip-sync breaks on the ordinary shots of any episode. Familiar Labs is one shot and the quality: translation first, reviewed; then the voice, the lips and the scene together, through the hard shots.

54.7%

FEWER TRANSLATION ERRORS · FAMILIAR LABS VS ELEVENLABS DUBBING V2

270%

ELEVENLABS MORE BACKGROUND-SOUND ERROR · THE LAUGHTER, THE MUSIC, THE AMBIENCE

+27.8%

FAMILIAR LABS CLOSER TO THE REAL SPEAKER

FAMILIAR LABS · ONE SHOT AND THE QUALITYFamiliar Labs vs Flawless TrueSync The ElevenLabs study The demos

What is the difference between localization and translation?

Translation is the words. Localization is what a culture needs from them.

The right terms, the slang, the names kept as said, a reference swapped for one the audience knows, what that culture avoids, the profanity level that fits. MLFM decides all of it, and in time the picture too, when it has to change.

Is MLFM generative AI?

No. IBM defines it this way:

Generative AI, sometimes called gen AI, is artificial intelligence (AI) that can create original content such as text, images, video, audio or software code in response to a user's prompt or request.
IBM, What is generative AI?

That is a strong tool for making new things. MLFM has no prompt, and nothing new is made up: it decides how to localize what is already there. The full position is in Why Familiar Labs Is Not Traditional Generative AI and Translation Is Not Generation.

Whose decisions are they?

Yours. Every decision the MLFM makes is recorded with the dub, in the decision record.

  1. TODAY

    Your creative directors review the translation decisions.

    The renderings with the reasoning behind each; they pick one before anything goes out.
  2. TODAY

    A Do Not Translate list and a one-sentence context travel with the job.

  3. IN THE WORKS

    The audio and visual decisions, on the same review.

  4. IN THE WORKS

    Decision memory.

    An override remembered for a creator applies first on the next dub, so a choice made once follows the next episode.

The full story is in The Decision Record; the planned API is in the decisions docs.

QUESTIONS

What does MLFM stand for?

MLFM stands for Multimodal Localization Foundation Model, a new paradigm and the first of its kind: one model that decides what to localize and how, across text, audio and visual. It picks the rendering of a line, what to translate in the audio, who on screen is speaking, and how the lips, the expressions and the gestures follow.

Is MLFM a video model?

Not in the usual sense. MLFM works on footage that already exists and decides how to localize it: the words, the voice, the lips and the expressions. The person on screen stays exactly who they are, in every language.

REFERENCES

  1. [1]IBM, What is generative AI?, read October 27, 2026
  2. [2]Our mission
  3. [3]Familiar Labs FAQ

Multimodal Translation

Familiar Labs builds generationally better dubbing with a Multimodal Localization Foundation Model (MLFM), the first foundation model for multimodal translation: one model decides what to localize across text, audio and visual, and shows you the choices. Your voice and lips, carried into new languages. They're a team of 7 with PhDs, backed by Samsung, RedBird, Gold House. Our mission

See the full benchmark →Talk with the team