Daily papers related to Image/Video/Multimodal Generation from cs.CV
January 26, 2026
Recent advances in video generation have produced models capable of synthesizing stunning visual content from simple text prompts. However, these models struggle to generate long-form, coherent narratives from high-level concepts like dialogue, revealing a ``semantic gap'' between a creative idea and its cinematic execution. To bridge this gap, we introduce a novel, end-to-end agentic framework for dialogue-to-cinematic-video generation. Central to our framework is ScripterAgent, a model trained to translate coarse dialogue into a fine-grained, executable cinematic script. To enable this, we construct ScriptBench, a new large-scale benchmark with rich multimodal context, annotated via an expert-guided pipeline. The generated script then guides DirectorAgent, which orchestrates state-of-the-art video models using a cross-scene continuous generation strategy to ensure long-horizon coherence. Our comprehensive evaluation, featuring an AI-powered CriticAgent and a new Visual-Script Alignment (VSA) metric, shows our framework significantly improves script faithfulness and temporal fidelity across all tested video models. Furthermore, our analysis uncovers a crucial trade-off in current SOTA models between visual spectacle and strict script adherence, providing valuable insights for the future of automated filmmaking.
TLDR: This paper introduces an agentic framework using a 'ScripterAgent' and 'DirectorAgent' to generate long-form videos from dialogue, addressing the semantic gap between creative ideas and cinematic execution. They also introduce a new benchmark and evaluation metric.
TLDR: 本文介绍了一种代理框架，使用'ScripterAgent'和'DirectorAgent'从对话生成长视频，解决了创意想法和电影制作之间的语义差距。他们还引入了一个新的基准和评估指标。
Read Paper (PDF)Controllable image generation is fundamental to the success of modern generative AI, yet it faces a critical trade-off between semantic fidelity and inference speed. The RemEdit diffusion-based framework addresses this trade-off with two synergistic innovations. First, for editing fidelity, we navigate the latent space as a Riemannian manifold. A mamba-based module efficiently learns the manifold's structure, enabling direct and accurate geodesic path computation for smooth semantic edits. This control is further refined by a dual-SLERP blending technique and a goal-aware prompt enrichment pass from a Vision-Language Model. Second, for additional acceleration, we introduce a novel task-specific attention pruning mechanism. A lightweight pruning head learns to retain tokens essential to the edit, enabling effective optimization without the semantic degradation common in content-agnostic approaches. RemEdit surpasses prior state-of-the-art editing frameworks while maintaining real-time performance under 50% pruning. Consequently, RemEdit establishes a new benchmark for practical and powerful image editing. Source code: https://www.github.com/eashanadhikarla/RemEdit.
TLDR: RemEdit introduces a diffusion-based image editing framework that achieves a better trade-off between editing fidelity and inference speed by using Riemannian geometry for semantic edits and task-specific attention pruning for acceleration. It claims state-of-the-art editing performance with real-time execution.
TLDR: RemEdit 提出了一个基于扩散模型的图像编辑框架，通过使用黎曼几何进行语义编辑和任务特定的注意力剪枝来加速，从而在编辑保真度和推理速度之间实现了更好的权衡。 它声称实现了最先进的编辑性能和实时执行。
Read Paper (PDF)Denoising-based diffusion transformers, despite their strong generation performance, suffer from inefficient training convergence. Existing methods addressing this issue, such as REPA (relying on external representation encoders) or SRA (requiring dual-model setups), inevitably incur heavy computational overhead during training due to external dependencies. To tackle these challenges, this paper proposes \textbf{\namex}, a lightweight intrinsic guidance framework for efficient diffusion training. \name leverages off-the-shelf pre-trained Variational Autoencoder (VAE) features: their reconstruction property ensures inherent encoding of visual priors like rich texture details, structural patterns, and basic semantic information. Specifically, \name aligns the intermediate latent features of diffusion transformers with VAE features via a lightweight projection layer, supervised by a feature alignment loss. This design accelerates training without extra representation encoders or dual-model maintenance, resulting in a simple yet effective pipeline. Extensive experiments demonstrate that \name improves both generation quality and training convergence speed compared to vanilla diffusion transformers, matches or outperforms state-of-the-art acceleration methods, and incurs merely 4\% extra GFLOPs with zero additional cost for external guidance models.
TLDR: The paper introduces VAE-REPA, a lightweight approach to accelerate diffusion model training by aligning intermediate latent features with pre-trained VAE features, achieving improved performance and faster convergence without significant computational overhead.
TLDR: 该论文介绍了VAE-REPA，一种轻量级的方法，通过将中间潜在特征与预训练的VAE特征对齐来加速扩散模型的训练，从而在不产生显著计算开销的情况下实现性能提升和更快的收敛。
Read Paper (PDF)Existing Subject-to-Video Generation (S2V) methods have achieved high-fidelity and subject-consistent video generation, yet remain constrained to single-view subject references. This limitation renders the S2V task reducible to an S2I + I2V pipeline, failing to exploit the full potential of video subject control. In this work, we propose and address the challenging Multi-View S2V (MV-S2V) task, which synthesizes videos from multiple reference views to enforce 3D-level subject consistency. Regarding the scarcity of training data, we first develop a synthetic data curation pipeline to generate highly customized synthetic data, complemented by a small-scale real-world captured dataset to boost the training of MV-S2V. Another key issue lies in the potential confusion between cross-subject and cross-view references in conditional generation. To overcome this, we further introduce Temporally Shifted RoPE (TS-RoPE) to distinguish between different subjects and distinct views of the same subject in reference conditioning. Our framework achieves superior 3D subject consistency w.r.t. multi-view reference images and high-quality visual outputs, establishing a new meaningful direction for subject-driven video generation. Our project page is available at this URL
TLDR: The paper introduces Multi-View Subject-to-Video (MV-S2V) generation, addressing the challenge of creating consistent videos from multiple subject viewpoints by using synthetic data and Temporally Shifted RoPE to differentiate subjects and views.
TLDR: 该论文介绍了多视角主体到视频 (MV-S2V) 生成，通过使用合成数据和时移 RoPE 来区分主体和视角，从而解决了从多个主体视角创建一致视频的难题。
Read Paper (PDF)Unified remote sensing multimodal models exhibit a pronounced spatial reversal curse: Although they can accurately recognize and describe object locations in images, they often fail to faithfully execute the same spatial relations during text-to-image generation, where such relations constitute core semantic information in remote sensing. Motivated by this observation, we propose Uni-RS, the first unified multimodal model tailored for remote sensing, to explicitly address the spatial asymmetry between understanding and generation. Specifically, we first introduce explicit Spatial-Layout Planning to transform textual instructions into spatial layout plans, decoupling geometric planning from visual synthesis. We then impose Spatial-Aware Query Supervision to bias learnable queries toward spatial relations explicitly specified in the instruction. Finally, we develop Image-Caption Spatial Layout Variation to expose the model to systematic geometry-consistent spatial transformations. Extensive experiments across multiple benchmarks show that our approach substantially improves spatial faithfulness in text-to-image generation, while maintaining strong performance on multimodal understanding tasks like image captioning, visual grounding, and VQA tasks.
TLDR: The paper introduces Uni-RS, a unified multimodal model specifically designed for remote sensing, which addresses the spatial reversal curse by explicitly managing spatial information during text-to-image generation, leading to improved spatial faithfulness.
TLDR: 该论文介绍了Uni-RS，一种专为遥感设计的统一多模态模型，通过在文本到图像生成过程中显式管理空间信息来解决空间反转问题，从而提高了空间保真度。
Read Paper (PDF)Real-world meal images often contain multiple food items, making reliable compositional food image generation important for applications such as image-based dietary assessment, where multi-food data augmentation is needed, and recipe visualization. However, modern text-to-image diffusion models struggle to generate accurate multi-food images due to object entanglement, where adjacent foods (e.g., rice and soup) fuse together because many foods do not have clear boundaries. To address this challenge, we introduce Prompt Grafting (PG), a training-free framework that combines explicit spatial cues in text with implicit layout guidance during sampling. PG runs a two-stage process where a layout prompt first establishes distinct regions and the target prompt is grafted once layout formation stabilizes. The framework enables food entanglement control: users can specify which food items should remain separated or be intentionally mixed by editing the arrangement of layouts. Across two food datasets, our method significantly improves the presence of target objects and provides qualitative evidence of controllable separation.
TLDR: The paper introduces Prompt Grafting (PG), a training-free framework to improve compositional text-to-image food generation by addressing the object entanglement problem in diffusion models, enabling controllable separation of food items.
TLDR: 本文介绍了一种名为Prompt Grafting (PG)的免训练框架，通过解决扩散模型中的对象纠缠问题，改进了组合文本到图像的食物生成，从而可以控制食物项的分离。
Read Paper (PDF)Diffusion Large Language Models (dLLMs) offer a compelling paradigm for natural language generation, leveraging parallel decoding and bidirectional attention to achieve superior global coherence compared to autoregressive models. While recent works have accelerated inference via KV cache reuse or heuristic decoding, they overlook the intrinsic inefficiencies within the block-wise diffusion process. Specifically, they suffer from spatial redundancy by modeling informative-sparse suffix regions uniformly and temporal inefficiency by applying fixed denoising schedules across all the decoding process. To address this, we propose Streaming-dLLM, a training-free framework that streamlines inference across both spatial and temporal dimensions. Spatially, we introduce attenuation guided suffix modeling to approximate the full context by pruning redundant mask tokens. Temporally, we employ a dynamic confidence aware strategy with an early exit mechanism, allowing the model to skip unnecessary iterations for converged tokens. Extensive experiments show that Streaming-dLLM achieves up to 68.2X speedup while maintaining generation quality, highlighting its effectiveness in diffusion decoding. The code is available at https://github.com/xiaoshideta/Streaming-dLLM.
TLDR: The paper proposes Streaming-dLLM, a training-free framework optimizing Diffusion Large Language Models (dLLMs) inference speed by pruning redundant tokens and dynamically adjusting denoising schedules, achieving up to 68.2X speedup.
TLDR: 该论文提出了Streaming-dLLM，一个无需训练的框架，通过剪枝冗余tokens和动态调整去噪计划来优化扩散大语言模型(dLLMs)的推理速度，实现了高达68.2倍的加速。
Read Paper (PDF)