Daily papers related to Image/Video/Multimodal Generation from cs.CV
February 10, 2026
We present Omni-Video 2, a scalable and computationally efficient model that connects pretrained multimodal large-language models (MLLMs) with video diffusion models for unified video generation and editing. Our key idea is to exploit the understanding and reasoning capabilities of MLLMs to produce explicit target captions to interpret user instructions. In this way, the rich contextual representations from the understanding model are directly used to guide the generative process, thereby improving performance on complex and compositional editing. Moreover, a lightweight adapter is developed to inject multimodal conditional tokens into pretrained text-to-video diffusion models, allowing maximum reuse of their powerful generative priors in a parameter-efficient manner. Benefiting from these designs, we scale up Omni-Video 2 to a 14B video diffusion model on meticulously curated training data with quality, supporting high quality text-to-video generation and various video editing tasks such as object removal, addition, background change, complex motion editing, \emph{etc.} We evaluate the performance of Omni-Video 2 on the FiVE benchmark for fine-grained video editing and the VBench benchmark for text-to-video generation. The results demonstrate its superior ability to follow complex compositional instructions in video editing, while also achieving competitive or superior quality in video generation tasks.
TLDR: Omni-Video 2 connects MLLMs with video diffusion models using a lightweight adapter, achieving state-of-the-art results on video generation and editing with a large 14B model.
TLDR: Omni-Video 2 通过轻量级适配器将多模态大型语言模型（MLLM）与视频扩散模型连接起来，通过一个14B的大型模型在视频生成和编辑方面取得了最先进的结果。
Read Paper (PDF)Audio is indispensable for real-world video, yet generation models have largely overlooked audio components. Current approaches to producing audio-visual content often rely on cascaded pipelines, which increase cost, accumulate errors, and degrade overall quality. While systems such as Veo 3 and Sora 2 emphasize the value of simultaneous generation, joint multimodal modeling introduces unique challenges in architecture, data, and training. Moreover, the closed-source nature of existing systems limits progress in the field. In this work, we introduce MOVA (MOSS Video and Audio), an open-source model capable of generating high-quality, synchronized audio-visual content, including realistic lip-synced speech, environment-aware sound effects, and content-aligned music. MOVA employs a Mixture-of-Experts (MoE) architecture, with a total of 32B parameters, of which 18B are active during inference. It supports IT2VA (Image-Text to Video-Audio) generation task. By releasing the model weights and code, we aim to advance research and foster a vibrant community of creators. The released codebase features comprehensive support for efficient inference, LoRA fine-tuning, and prompt enhancement.
TLDR: MOVA is an open-source, 32B parameter Mixture-of-Experts model for generating high-quality, synchronized audio-visual content from image-text prompts, including lip-synced speech and environment-aware sound effects.
TLDR: MOVA是一个开源的、拥有320亿参数的混合专家模型，能够根据图像-文本提示生成高质量、同步的视听内容，包括唇音同步的语音和环境感知的音效。
Read Paper (PDF)Video generation is rapidly evolving towards unified audio-video generation. In this paper, we present ALIVE, a generation model that adapts a pretrained Text-to-Video (T2V) model to Sora-style audio-video generation and animation. In particular, the model unlocks the Text-to-Video&Audio (T2VA) and Reference-to-Video&Audio (animation) capabilities compared to the T2V foundation models. To support the audio-visual synchronization and reference animation, we augment the popular MMDiT architecture with a joint audio-video branch which includes TA-CrossAttn for temporally-aligned cross-modal fusion and UniTemp-RoPE for precise audio-visual alignment. Meanwhile, a comprehensive data pipeline consisting of audio-video captioning, quality control, etc., is carefully designed to collect high-quality finetuning data. Additionally, we introduce a new benchmark to perform a comprehensive model test and comparison. After continue pretraining and finetuning on million-level high-quality data, ALIVE demonstrates outstanding performance, consistently outperforming open-source models and matching or surpassing state-of-the-art commercial solutions. With detailed recipes and benchmarks, we hope ALIVE helps the community develop audio-video generation models more efficiently. Official page: https://github.com/FoundationVision/Alive.
TLDR: ALIVE is a new audio-video generation model built upon a T2V foundation model, enabling Text-to-Video&Audio and Reference-to-Video&Audio capabilities, outperforming open-source models and matching/surpassing commercial alternatives.
TLDR: ALIVE是一个新的音视频生成模型，它建立在T2V基础模型之上，实现了文本到视频音频和参考视频到视频音频的功能，性能优于开源模型，并且可以与商业替代方案相媲美甚至超越。
Read Paper (PDF)Flow matching models have recently emerged as an efficient alternative to diffusion, especially for text-guided image generation and editing, offering faster inference through continuous-time dynamics. However, existing flow-based editors predominantly support global or single-instruction edits and struggle with multi-instance scenarios, where multiple parts of a reference input must be edited independently without semantic interference. We identify this limitation as a consequence of globally conditioned velocity fields and joint attention mechanisms, which entangle concurrent edits. To address this issue, we introduce Instance-Disentangled Attention, a mechanism that partitions joint attention operations, enforcing binding between instance-specific textual instructions and spatial regions during velocity field estimation. We evaluate our approach on both natural image editing and a newly introduced benchmark of text-dense infographics with region-level editing instructions. Experimental results demonstrate that our approach promotes edit disentanglement and locality while preserving global output coherence, enabling single-pass, instance-level editing.
TLDR: The paper introduces Instance-Disentangled Attention to improve flow matching models for multi-instance image editing, addressing the limitation of existing methods in handling independent edits of multiple parts of an image. Experimental results show improved edit disentanglement and locality with global coherence.
TLDR: 该论文介绍了一种实例解耦注意力机制，以改进用于多实例图像编辑的流匹配模型。该方法解决了现有方法在处理图像多个部分独立编辑方面的局限性。实验结果表明，该方法在保持全局一致性的前提下，提高了编辑的解耦性和局部性。
Read Paper (PDF)Hypothesis. Artificial general intelligence is, at its core, a compression problem. Effective compression demands resonance: deep learning scales best when its architecture aligns with the fundamental structure of the data. These are the fundamental principles. Yet, modern vision architectures have strayed from these truths: visual signals are highly redundant, while discriminative information, the surprise, is sparse. Current models process dense pixel grids uniformly, wasting vast compute on static background rather than focusing on the predictive residuals that define motion and meaning. We argue that to solve visual understanding, we must align our architectures with the information-theoretic principles of video, i.e., Codecs. Method. OneVision-Encoder encodes video by compressing predictive visual structure into semantic meaning. By adopting Codec Patchification, OV-Encoder abandons uniform computation to focus exclusively on the 3.1%-25% of regions rich in signal entropy. To unify spatial and temporal reasoning under irregular token layouts, OneVision-Encoder employs a shared 3D RoPE and is trained with a large-scale cluster discrimination objective over more than one million semantic concepts, jointly capturing object permanence and motion dynamics. Evidence. The results validate our core hypothesis: efficiency and accuracy are not a trade-off; they are positively correlated. When integrated into LLM, it consistently outperforms strong vision backbones such as Qwen3-ViT and SigLIP2 across 16 image, video, and document understanding benchmarks, despite using substantially fewer visual tokens and pretraining data. Notably, on video understanding tasks, OV-Encoder achieves an average improvement of 4.1% over Qwen3-ViT. Codec-aligned, patch-level sparsity is a foundational principle, enabling OV-Encoder as a scalable engine for next-generation visual generalists.
TLDR: The paper introduces OneVision-Encoder (OV-Encoder), a video encoder leveraging codec-aligned, patch-level sparsity to improve efficiency and accuracy across various multimodal understanding tasks, demonstrating superior performance compared to state-of-the-art vision backbones.
TLDR: 本文介绍了一种名为OneVision-Encoder (OV-Encoder) 的视频编码器，它利用编解码器对齐的patch级稀疏性来提高在各种多模态理解任务中的效率和准确性，并展示了优于最先进的视觉骨干网络的性能。
Read Paper (PDF)Text-to-motion generation, a rapidly evolving field in computer vision, aims to produce realistic and text-aligned motion sequences. Current methods primarily focus on spatial-temporal modeling or independent frequency domain analysis, lacking a unified framework for joint optimization across spatial, temporal, and frequency domains. This limitation hinders the model's ability to leverage information from all domains simultaneously, leading to suboptimal generation quality. Additionally, in motion generation frameworks, motion-irrelevant cues caused by noise are often entangled with features that contribute positively to generation, thereby leading to motion distortion. To address these issues, we propose Tri-Domain Causal Text-to-Motion Generation (TriC-Motion), a novel diffusion-based framework integrating spatial-temporal-frequency-domain modeling with causal intervention. TriC-Motion includes three core modeling modules for domain-specific modeling, namely Temporal Motion Encoding, Spatial Topology Modeling, and Hybrid Frequency Analysis. After comprehensive modeling, a Score-guided Tri-domain Fusion module integrates valuable information from the triple domains, simultaneously ensuring temporal consistency, spatial topology, motion trends, and dynamics. Moreover, the Causality-based Counterfactual Motion Disentangler is meticulously designed to expose motion-irrelevant cues to eliminate noise, disentangling the real modeling contributions of each domain for superior generation. Extensive experimental results validate that TriC-Motion achieves superior performance compared to state-of-the-art methods, attaining an outstanding R@1 of 0.612 on the HumanML3D dataset. These results demonstrate its capability to generate high-fidelity, coherent, diverse, and text-aligned motion sequences. Code is available at: https://caoyiyang1105.github.io/TriC-Motion/.
TLDR: The paper introduces TriC-Motion, a novel diffusion-based framework for text-to-motion generation that integrates spatial-temporal-frequency domain modeling with causal intervention, achieving state-of-the-art results on HumanML3D.
TLDR: 该论文介绍了 TriC-Motion，一个新颖的基于扩散模型的文本到运动生成框架，它集成了空间-时间-频率域建模与因果干预，在 HumanML3D 数据集上取得了最先进的结果。
Read Paper (PDF)Recent advances in diffusion models have significantly improved image editing. However, challenges persist in handling geometric transformations, such as translation, rotation, and scaling, particularly in complex scenes. Existing approaches suffer from two main limitations: (1) difficulty in achieving accurate geometric editing of object translation, rotation, and scaling; (2) inadequate modeling of intricate lighting and shadow effects, leading to unrealistic results. To address these issues, we propose GeoEdit, a framework that leverages in-context generation through a diffusion transformer module, which integrates geometric transformations for precise object edits. Moreover, we introduce Effects-Sensitive Attention, which enhances the modeling of intricate lighting and shadow effects for improved realism. To further support training, we construct RS-Objects, a large-scale geometric editing dataset containing over 120,000 high-quality image pairs, enabling the model to learn precise geometric editing while generating realistic lighting and shadows. Extensive experiments on public benchmarks demonstrate that GeoEdit consistently outperforms state-of-the-art methods in terms of visual quality, geometric accuracy, and realism.
TLDR: The paper introduces GeoEdit, a diffusion transformer-based framework with Effects-Sensitive Attention, and a new dataset (RS-Objects) to improve geometric image editing accuracy and realism, specifically addressing object translation, rotation, scaling, lighting, and shadow effects.
TLDR: 该论文介绍了GeoEdit，一个基于扩散Transformer的框架，具有效果敏感注意力机制，并提出了一个新的数据集（RS-Objects），以提高图像几何编辑的准确性和真实感，特别针对对象平移、旋转、缩放、光照和阴影效果。
Read Paper (PDF)To elicit capabilities for addressing complex and implicit visual requirements, recent unified multimodal models increasingly adopt chain-of-thought reasoning to guide image generation. However, the actual effect of reasoning on visual synthesis remains unclear. We present UReason, a diagnostic benchmark for reasoning-driven image generation that evaluates whether reasoning can be faithfully executed in pixels. UReason contains 2,000 instances across five task families: Code, Arithmetic, Spatial, Attribute, and Text reasoning. To isolate the role of reasoning traces, we introduce an evaluation framework comparing direct generation, reasoning-guided generation, and de-contextualized generation which conditions only on the refined prompt. Across eight open-source unified models, we observe a consistent Reasoning Paradox: Reasoning traces generally improve performance over direct generation, yet retaining intermediate thoughts as conditioning context often hinders visual synthesis, and conditioning only on the refined prompt yields substantial gains. Our analysis suggests that the bottleneck lies in contextual interference rather than insufficient reasoning capacity. UReason provides a principled testbed for studying reasoning in unified models and motivates future methods that effectively integrate reasoning for visual generation while mitigating interference.
TLDR: The paper introduces UReason, a benchmark for evaluating reasoning in unified multimodal models for image generation, revealing a 'Reasoning Paradox' where reasoning traces improve generation but hinder performance when used as conditioning context.
TLDR: 该论文介绍了 UReason，一个用于评估统一多模态模型中图像生成推理能力的基准。研究揭示了“推理悖论”，即推理过程可以改善生成效果，但将其用作上下文会阻碍性能。
Read Paper (PDF)Image reconstruction and image synthesis are important for handling incomplete multimodal imaging data, but existing methods require various task-specific models, complicating training and deployment workflows. We introduce Any2all, a unified framework that addresses this limitation by formulating these disparate tasks as a single virtual inpainting problem. We train a single, unconditional diffusion model on the complete multimodal data stack. This model is then adapted at inference time to ``inpaint'' all target modalities from any combination of inputs of available clean images or noisy measurements. We validated Any2all on a PET/MR/CT brain dataset. Our results show that Any2all can achieve excellent performance on both multimodal reconstruction and synthesis tasks, consistently yielding images with competitive distortion-based performance and superior perceptual quality over specialized methods.
TLDR: This paper introduces Any2all, a unified framework using denoising diffusion models for both multimodal image reconstruction and synthesis, simplifying the process compared to task-specific models.
TLDR: 本文介绍了一个名为Any2all的统一框架，该框架使用去噪扩散模型进行多模态图像重建和合成，与特定任务的模型相比，简化了流程。
Read Paper (PDF)This work presents a systematic investigation into modernizing Vision Transformer backbones by leveraging architectural advancements from the past five years. While preserving the canonical Attention-FFN structure, we conduct a component-wise refinement involving normalization, activation functions, positional encoding, gating mechanisms, and learnable tokens. These updates form a new generation of Vision Transformers, which we call ViT-5. Extensive experiments demonstrate that ViT-5 consistently outperforms state-of-the-art plain Vision Transformers across both understanding and generation benchmarks. On ImageNet-1k classification, ViT-5-Base reaches 84.2\% top-1 accuracy under comparable compute, exceeding DeiT-III-Base at 83.8\%. ViT-5 also serves as a stronger backbone for generative modeling: when plugged into an SiT diffusion framework, it achieves 1.84 FID versus 2.06 with a vanilla ViT backbone. Beyond headline metrics, ViT-5 exhibits improved representation learning and favorable spatial reasoning behavior, and transfers reliably across tasks. With a design aligned with contemporary foundation-model practices, ViT-5 offers a simple drop-in upgrade over vanilla ViT for mid-2020s vision backbones.
TLDR: ViT-5 is a modernized Vision Transformer architecture leveraging recent advancements, achieving state-of-the-art performance in image classification and serving as a better backbone for generative models compared to vanilla ViTs.
TLDR: ViT-5 是一种现代化的视觉 Transformer 架构，利用了最新的进展。与原始 ViT 相比，它在图像分类中实现了最先进的性能，并且是生成模型的更好骨干。
Read Paper (PDF)Text-guided image editing aims to modify specific regions according to the target prompt while preserving the identity of the source image. Recent methods exploit explicit binary masks to constrain editing, but hard mask boundaries introduce artifacts and reduce editability. To address these issues, we propose FusionEdit, a training-free image editing framework that achieves precise and controllable edits. First, editing and preserved regions are automatically identified by measuring semantic discrepancies between source and target prompts. To mitigate boundary artifacts, FusionEdit performs distance-aware latent fusion along region boundaries to yield the soft and accurate mask, and employs a total variation loss to enforce smooth transitions, obtaining natural editing results. Second, FusionEdit leverages AdaIN-based modulation within DiT attention layers to perform a statistical attention fusion in the editing region, enhancing editability while preserving global consistency with the source image. Extensive experiments demonstrate that our FusionEdit significantly outperforms state-of-the-art methods. Code is available at \href{https://github.com/Yvan1001/FusionEdit}{https://github.com/Yvan1001/FusionEdit}.
TLDR: FusionEdit is a training-free image editing framework that uses semantic fusion and attention modulation to achieve precise and controllable edits based on text prompts while minimizing artifacts and preserving image identity.
TLDR: FusionEdit是一个无需训练的图像编辑框架，通过语义融合和注意力调制，根据文本提示实现精确可控的图像编辑，同时最大限度地减少伪影并保持图像的原始特征。
Read Paper (PDF)We introduce FLAG-4D, a novel framework for generating novel views of dynamic scenes by reconstructing how 3D Gaussian primitives evolve through space and time. Existing methods typically rely on a single Multilayer Perceptron (MLP) to model temporal deformations, and they often struggle to capture complex point motions and fine-grained dynamic details consistently over time, especially from sparse input views. Our approach, FLAG-4D, overcomes this by employing a dual-deformation network that dynamically warps a canonical set of 3D Gaussians over time into new positions and anisotropic shapes. This dual-deformation network consists of an Instantaneous Deformation Network (IDN) for modeling fine-grained, local deformations and a Global Motion Network (GMN) for capturing long-range dynamics, refined through mutual learning. To ensure these deformations are both accurate and temporally smooth, FLAG-4D incorporates dense motion features from a pretrained optical flow backbone. We fuse these motion cues from adjacent timeframes and use a deformation-guided attention mechanism to align this flow information with the current state of each evolving 3D Gaussian. Extensive experiments demonstrate that FLAG-4D achieves higher-fidelity and more temporally coherent reconstructions with finer detail preservation than state-of-the-art methods.
TLDR: The paper introduces FLAG-4D, a dual-deformation network for 4D reconstruction that uses a novel approach to model temporal deformations of 3D Gaussians using optical flow to improve fidelity and temporal coherence in dynamic scene reconstruction. This tackles issues with consistency over time, especially given sparse views.
TLDR: 该论文介绍了FLAG-4D，一种用于4D重建的双重形变网络，它使用一种新方法来模拟3D高斯函数的时序形变，通过光流来提高动态场景重建的保真度和时间连贯性。这解决了时间上的一致性问题，尤其是在给定稀疏视图的情况下。
Read Paper (PDF)In this paper, we focus on motion discrete tokenization, which converts raw motion into compact discrete tokens--a process proven crucial for efficient motion generation. In this paradigm, increasing the number of tokens is a common approach to improving motion reconstruction quality, but more tokens make it more difficult for generative models to learn. To maintain high reconstruction quality while reducing generation complexity, we propose leveraging language to achieve efficient motion tokenization, which we term Language-Guided Tokenization (LG-Tok). LG-Tok aligns natural language with motion at the tokenization stage, yielding compact, high-level semantic representations. This approach not only strengthens both tokenization and detokenization but also simplifies the learning of generative models. Furthermore, existing tokenizers predominantly adopt convolutional architectures, whose local receptive fields struggle to support global language guidance. To this end, we propose a Transformer-based Tokenizer that leverages attention mechanisms to enable effective alignment between language and motion. Additionally, we design a language-drop scheme, in which language conditions are randomly removed during training, enabling the detokenizer to support language-free guidance during generation. On the HumanML3D and Motion-X generation benchmarks, LG-Tok achieves Top-1 scores of 0.542 and 0.582, outperforming state-of-the-art methods (MARDM: 0.500 and 0.528), and with FID scores of 0.057 and 0.088, respectively, versus 0.114 and 0.147. LG-Tok-mini uses only half the tokens while maintaining competitive performance (Top-1: 0.521/0.588, FID: 0.085/0.071), validating the efficiency of our semantic representations.
TLDR: This paper introduces a language-guided tokenization method (LG-Tok) for human motion generation, using a Transformer-based tokenizer and language-drop scheme to achieve efficient and high-quality motion representation, outperforming existing methods on standard benchmarks.
TLDR: 本文提出了一种用于人体运动生成的语言引导标记化方法（LG-Tok），它使用基于Transformer的标记器和语言丢弃方案来实现高效和高质量的运动表示，并在标准基准测试中优于现有方法。
Read Paper (PDF)We present PEGAsus, a new framework capable of generating Personalized 3D shapes by learning shape concepts at both Geometry and Appearance levels. First, we formulate 3D shape personalization as extracting reusable, category-agnostic geometric and appearance attributes from reference shapes, and composing these attributes with text to generate novel shapes. Second, we design a progressive optimization strategy to learn shape concepts at both the geometry and appearance levels, decoupling the shape concept learning process. Third, we extend our approach to region-wise concept learning, enabling flexible concept extraction, with context-aware and context-free losses. Extensive experimental results show that PEGAsus is able to effectively extract attributes from a wide range of reference shapes and then flexibly compose these concepts with text to synthesize new shapes. This enables fine-grained control over shape generation and supports the creation of diverse, personalized results, even in challenging cross-category scenarios. Both quantitative and qualitative experiments demonstrate that our approach outperforms existing state-of-the-art solutions.
TLDR: PEGAsus is a new framework for personalized 3D shape generation based on geometric and appearance attributes extracted from reference shapes and composed with text prompts, outperforming existing methods.
TLDR: PEGAsus是一个新的个性化3D形状生成框架，它基于从参考形状中提取的几何和外观属性，并结合文本提示，其性能优于现有方法。
Read Paper (PDF)