New Pipelines Cosmos 3 Cosmos 3 is NVIDIA's unified world foundation model (WFM) for Physical AI — a single omni-model built on a Mixture-of-Transformers (MoT) architecture that combines world generation, physical reasoning, and action generation, replacing the separate Predict, Reason, and Transfer models from earlier Cosmos releases. A single Cosmos3OmniTransformer runs a Qwen-style language mod…
New Pipelines LLaDA2 LLaDA2 is a family of discrete diffusion language models that generate text through block-wise iterative refinement. Instead of autoregressive token-by-token generation, LLaDA2 starts with a fully masked sequence and progressively unmasks tokens by confidence over multiple refinement steps. PR: #13226 Docs: https://huggingface.co/docs/diffusers/main/api/pipelines/llada2 Nucleu…
Fix for loading ModularPipelines with AutoModel type hints in their modular_model_index.json #13271 Fix Flux Klein LoRA loading #13313 Fix unguarded torchvision import in Cosmos Predict 2.5 #13321
Modular Diffusers Modular Diffusers introduces a new way to build diffusion pipelines by composing reusable blocks. Instead of writing entire pipelines from scratch, you can now mix and match building blocks to create custom workflows tailored to your specific needs! This complements the existing DiffusionPipeline class, providing a more flexible way to create custom diffusion pipelines. Find more…
The release features a number of new image and video pipelines, a new caching method, a new training script, new kernels - powered attention backends, and more. It is quite packed with a lot of new stuff, so make sure you read the release notes fully 🚀 New image pipelines Flux2: Flux2 is the latest generation of image generation and editing model from Black Forest Labs. It’s capable of taking mult…
All commits Release: v0.35.1-patch by @sayakpaul (direct commit on v0.35.2-patch) handle offload_state_dict when initing transformers models by @sayakpaul in #12438 [CI] Fix TRANSFORMERS_FLAX_WEIGHTS_NAME import issue by @DN6 in #12354 Fix PyTorch 2.3.1 compatibility: add version guard for torch.library.… by @Aishwarya0811 in #12206 fix scale_shift_factor being on cpu for wan and ltx by @vladmandi…
Thanks to @naykun for the following PRs that improve Qwen-Image Edit: #12188 #12190
This release comes packed with new image generation and editing pipelines, a new video pipeline, new training scripts, quality-of-life improvements, and much more. Read the rest of the release notes fully to not miss out on the fun stuff. New pipelines 🧨 We welcomed new pipelines in this release: Wan 2.2 Flux-Kontext Qwen-Image Qwen-Image-Edit Wan 2.2 📹 This update to Wan provides significant impr…
📹 New video generation pipelines Wan VACE Wan VACE supports various generation techniques which achieve controllable video generation. It comes in two variants: a 1.3B model for fast iteration & prototyping, and a 14B for high quality generation. Some of the capabilities include: Control to Video (Depth, Pose, Sketch, Flow, Grayscale, Scribble, Layout, Boundary Box, etc.). Recommended library for…
All commits fix ftfy import for wan pipelines by @yiyixuxu in #11262
New Pipelines for Video Generation Wan 2.1 Wan2.1 is a comprehensive and open suite of video foundation models that pushes the boundaries of video generation. The model release includes 4 different model variants and three different pipelines for Text to Video, Image to Video and Video to Video. Wan-AI/Wan2.1-T2V-1.3B-Diffusers Wan-AI/Wan2.1-T2V-14B-Diffusers Wan-AI/Wan2.1-I2V-14B-480P-Diffusers W…