The Research Behind Familiar: The Published Papers

The first joint audio-visual generation model: speech and facial motion come out of one flow-matching model, so the voice and the face can never drift apart.

Audio-to-video generation that lands motion on the beat, with motion-aware training and Audio Sync Guidance, plus CycleSync, a new metric for measuring sync.

Emotion control for text-to-speech that changes over time: a frozen speech model plus a trainable ControlNet copy, with zero-shot voice cloning kept intact.

Real-time human animation at 20 frames per second that stays stable for an hour, using Neighbor Forcing and ConvKV memory to stop the drift that kills long generations.

Stage-aware sampling that swaps between large and small models across denoising stages: up to 1.65× faster video generation at large-model fidelity.

Sharper classifier-free guidance: an SVD-based refinement of the unconditional score that steers sampling back toward the data manifold.

Straighter rectified flows trained with inversions of real data, so faster few-step sampling stays anchored to the true distribution.

Distills diffusion models down to 8 steps with an external guide: about 1% of parameters trained, no classifier-free guidance needed at inference.

Turns a frozen text-to-image model into a video generator with a motion module and new regularizers, without retraining the base model.

Found the semantic latent space hiding inside frozen diffusion models, h-space, and edited images through it with no training.

Training-free content injection through the same h-space: move the content of one image into another inside a frozen diffusion model.

Maps the geometry of the diffusion latent space with Riemannian pullback metrics, showing when and where edits actually work across timesteps.

Style without content leakage: negative visual query guidance keeps a reference image's style and leaves its objects behind.

Training-free style control: swap a reference image's self-attention into the generation and the output follows its style.

Removes on-screen text from video and restores what was behind it, end to end, with no OCR and no masks at inference.

The first benchmark for egocentric vision at night: day-night aligned videos that expose where models fail in the dark.

Retrieval-augmented image generation: real photos pulled in as references through self-reflective contrastive learning, so unfamiliar objects render true to life.

Presents Pano-R1, a panoramic reasoning model trained with reinforcement learning on CFpano, the first benchmark for question answering across correlated 360° views.

Puts a unified model's understanding to work during generation: the model reasons about the prompt first, then generates with that reasoning infused.

One balanced embedding space that binds seven modalities, image, text, audio, video and more, aligned to LLM-augmented class centers.