Daily papers related to Image/Video/Multimodal Generation from cs.CV
June 24, 2025
In this work, we introduce OmniGen2, a versatile and open-source generative model designed to provide a unified solution for diverse generation tasks, including text-to-image, image editing, and in-context generation. Unlike OmniGen v1, OmniGen2 features two distinct decoding pathways for text and image modalities, utilizing unshared parameters and a decoupled image tokenizer. This design enables OmniGen2 to build upon existing multimodal understanding models without the need to re-adapt VAE inputs, thereby preserving the original text generation capabilities. To facilitate the training of OmniGen2, we developed comprehensive data construction pipelines, encompassing image editing and in-context generation data. Additionally, we introduce a reflection mechanism tailored for image generation tasks and curate a dedicated reflection dataset based on OmniGen2. Despite its relatively modest parameter size, OmniGen2 achieves competitive results on multiple task benchmarks, including text-to-image and image editing. To further evaluate in-context generation, also referred to as subject-driven tasks, we introduce a new benchmark named OmniContext. OmniGen2 achieves state-of-the-art performance among open-source models in terms of consistency. We will release our models, training code, datasets, and data construction pipeline to support future research in this field. Project Page: https://vectorspacelab.github.io/OmniGen2; GitHub Link: https://github.com/VectorSpaceLab/OmniGen2
TLDR: OmniGen2 is a new open-source multimodal generative model that unifies text-to-image, image editing, and in-context generation using separate decoding pathways and achieves competitive performance, especially in subject-driven image generation consistency.
TLDR: OmniGen2是一个新的开源多模态生成模型，它统一了文本到图像、图像编辑和上下文生成，使用单独的解码路径，并实现了具有竞争力的性能，尤其是在受主体驱动的图像生成一致性方面。
Read Paper (PDF)This paper presents a multimodal framework that attempts to unify visual understanding and generation within a shared discrete semantic representation. At its core is the Text-Aligned Tokenizer (TA-Tok), which converts images into discrete tokens using a text-aligned codebook projected from a large language model's (LLM) vocabulary. By integrating vision and text into a unified space with an expanded vocabulary, our multimodal LLM, Tar, enables cross-modal input and output through a shared interface, without the need for modality-specific designs. Additionally, we propose scale-adaptive encoding and decoding to balance efficiency and visual detail, along with a generative de-tokenizer to produce high-fidelity visual outputs. To address diverse decoding needs, we utilize two complementary de-tokenizers: a fast autoregressive model and a diffusion-based model. To enhance modality fusion, we investigate advanced pre-training tasks, demonstrating improvements in both visual understanding and generation. Experiments across benchmarks show that Tar matches or surpasses existing multimodal LLM methods, achieving faster convergence and greater training efficiency. Code, models, and data are available at https://tar.csuhan.com
TLDR: The paper introduces Tar, a multimodal LLM that unifies visual understanding and generation using a text-aligned tokenizer, demonstrating improved performance and efficiency across benchmarks.
TLDR: 本文介绍了一种多模态LLM名为Tar，它使用文本对齐的tokenizer统一了视觉理解和生成，并在多个基准测试中展示了改进的性能和效率。
Read Paper (PDF)Customizing image generation remains a core challenge in controllable image synthesis. For single-concept generation, maintaining both identity preservation and prompt alignment is challenging. In multi-concept scenarios, relying solely on a prompt without additional conditions like layout boxes or semantic masks, often leads to identity loss and concept omission. In this paper, we introduce ShowFlow, a comprehensive framework designed to tackle these challenges. We propose ShowFlow-S for single-concept image generation, and ShowFlow-M for handling multiple concepts. ShowFlow-S introduces a KronA-WED adapter, which integrates a Kronecker adapter with weight and embedding decomposition, and employs a disentangled learning approach with a novel attention regularization objective to enhance single-concept generation. Building on this foundation, ShowFlow-M directly reuses the learned models from ShowFlow-S to support multi-concept generation without extra conditions, incorporating a Subject-Adaptive Matching Attention (SAMA) and a layout consistency strategy as the plug-and-play module. Extensive experiments and user studies validate ShowFlow's effectiveness, highlighting its potential in real-world applications like advertising and virtual dressing.
TLDR: The paper introduces ShowFlow, a framework comprising ShowFlow-S for single-concept and ShowFlow-M for multi-concept image generation, addressing identity preservation and prompt alignment without requiring explicit conditions.
TLDR: 该论文介绍了 ShowFlow，一个包含 ShowFlow-S（用于单概念图像生成）和 ShowFlow-M（用于多概念图像生成）的框架，旨在解决身份保持和提示对齐问题，而无需显式条件。
Read Paper (PDF)Autoregressive conditional image generation models have emerged as a dominant paradigm in text-to-image synthesis. These methods typically convert images into one-dimensional token sequences and leverage the self-attention mechanism, which has achieved remarkable success in natural language processing, to capture long-range dependencies, model global context, and ensure semantic coherence. However, excessively long contexts during inference lead to significant memory overhead caused by KV-cache and computational delays. To alleviate these challenges, we systematically analyze how global semantics, spatial layouts, and fine-grained textures are formed during inference, and propose a novel training-free context optimization method called Adaptive Dynamic Sparse Attention (ADSA). Conceptually, ADSA dynamically identifies historical tokens crucial for maintaining local texture consistency and those essential for ensuring global semantic coherence, thereby efficiently streamlining attention computation. Additionally, we introduce a dynamic KV-cache update mechanism tailored for ADSA, reducing GPU memory consumption during inference by approximately $50\%$. Extensive qualitative and quantitative experiments demonstrate the effectiveness and superiority of our approach in terms of both generation quality and resource efficiency.
TLDR: The paper introduces Adaptive Dynamic Sparse Attention (ADSA), a training-free method for optimizing attention computation in autoregressive image generation, significantly reducing memory overhead and computational delays during inference by dynamically selecting crucial historical tokens.
TLDR: 该论文介绍了自适应动态稀疏注意力 (ADSA)，一种用于优化自回归图像生成中注意力计算的免训练方法，通过动态选择关键的历史 token，显著降低了推理期间的内存开销和计算延迟。
Read Paper (PDF)