T3Mermaid to ASCII art (mermaid-ascii)T3Kimi K3, and what we can still learn from the pelican benchmarkT3Firefox in WebAssemblyT3Spot birds not golfT3[AINews] Kimi K3 2.8T-A50B: the largest open model ever released; Opus 4.8-class at Sonnet 5 pricingT1From physical surfaces to human-centric heat stress: LST and UTCI heat mapping reveals nonlinear effects of urban morphologyT1DualHNIE: Dual-Channel Hypergraph Learning for Node Importance Estimation in Heterogeneous Knowledge GraphsT1GenTL: A General Transfer Learning Model for Building Thermal DynamicsT1A short review on the maximum clique problem algorithms with classical, AI, and quantum methodsT1Man, Machine, and Masterpiece: Artistic Ownership in the AI EraT1HABIB_TAZ at SemEval-2026 Task 11: Disentangling Formal Logic from Content via Synthetic Training and Multi-Objective OptimizationT1How Well Does AI-Generated Feedback Work? Intrinsic and Extrinsic Evaluation across more than 20,000 EFL Essay DraftsT3Mermaid to ASCII art (mermaid-ascii)T3Kimi K3, and what we can still learn from the pelican benchmarkT3Firefox in WebAssemblyT3Spot birds not golfT3[AINews] Kimi K3 2.8T-A50B: the largest open model ever released; Opus 4.8-class at Sonnet 5 pricingT1From physical surfaces to human-centric heat stress: LST and UTCI heat mapping reveals nonlinear effects of urban morphologyT1DualHNIE: Dual-Channel Hypergraph Learning for Node Importance Estimation in Heterogeneous Knowledge GraphsT1GenTL: A General Transfer Learning Model for Building Thermal DynamicsT1A short review on the maximum clique problem algorithms with classical, AI, and quantum methodsT1Man, Machine, and Masterpiece: Artistic Ownership in the AI EraT1HABIB_TAZ at SemEval-2026 Task 11: Disentangling Formal Logic from Content via Synthetic Training and Multi-Objective OptimizationT1How Well Does AI-Generated Feedback Work? Intrinsic and Extrinsic Evaluation across more than 20,000 EFL Essay Drafts

Diffusion Models for Video Generation

Named practitioner synthesis

#235

T3digested

Canonical URL: https://lilianweng.github.io/posts/2024-04-12-diffusion-video/
Topic: Practitioner Notes
First seen: 2026-07-16 19:08:00
Last seen: 2026-07-16 19:08:00

Source raw items (1)

Blog / Newsletter2026-07-16 19:07:22
Diffusion Models for Video Generation
<p><a href="https://lilianweng.github.io/posts/2021-07-11-diffusion-models/">Diffusion models</a> have demonstrated strong results on image synthesis in past years. Now the research community has started working on a harder task—using it for video generation. The task itself is a superset of the image case, since an image is a video of 1 frame, and it is much more challenging because:</p> <ol> <li>It has extra requirements on temporal consistency across frames in time, which naturally demands more world knowledge to be encoded into the model.</li> <li>In comparison to text or images, it is more difficult to collect large amounts of high-quality, high-dimensional video data, let along text-video pairs.</li> </ol> <blockquote> <p><br /><b> 🥑 Required Pre-read: Please make sure you have read the previous blog on <a href="https://lilianweng.github.io/posts/2021-07-11-diffusion-models/">“What are Diffusion Models?”</a> for image generation before continue here. </b><br /><