Structural gap audit for the next Video Generation 101 expansion¶
Audit date: 2026-08-30 (Asia/Shanghai). This record explains why three new chapters were added and how they must be integrated without erasing task or evidence boundaries.
1. Review question¶
Which active video-generation research axes are already mentioned in the repository but still lack a dedicated chapter that can answer all of the following?
- What is the exact task or optimization contract?
- Which technical routes are genuinely different?
- Which works changed the contract or evidence standard?
- What is formally published, only a preprint, only an artifact, or only a product claim?
- What minimum reproduction and falsifier would test the main claim?
The audit is structural rather than keyword-only. A subject was not considered covered merely because several paper names appeared in a timeline or reading list.
2. Repository audit¶
Read-only checks used rg --files docs, the current coverage matrix, and targeted searches for audio-video, preference, reward, DPO, RL, camera, trajectory, pose, depth, ControlNet, MotionCtrl and CameraCtrl.
| Candidate axis | Existing coverage before this batch | Missing contract | Decision |
|---|---|---|---|
| Native joint audio-video | Scattered in T2V, foundation models, timeline and reading route | No chapter separates video-to-audio, staged generation, coupled dual streams, shared joint denoising, product output and streaming AV | Add a task chapter |
| Video post-training and alignment | Scattered in the mechanism overview, datasets, evaluation and a combined few-step reading route | No chapter separates SFT, reward modeling, DPO/RWR, policy-gradient RL, inference guidance and distillation, or treats reward leakage and denoising credit assignment as first-class risks | Add a mechanism chapter |
| Fine-grained controllable generation | The taxonomy lists masks, trajectories and cameras, but the repository had no dedicated method lineage | No chapter organizes control signal, coordinate system, injection point, source-preservation contract, conflict handling or control-specific evaluation | Add a task chapter |
3. Inclusion and exclusion rules¶
A new chapter was included only when the axis had:
- a distinct input/output or optimization contract;
- at least three independently identifiable primary works or official releases;
- evaluation failures that cannot be diagnosed by generic T2V quality metrics;
- enough current activity that folding it into a paragraph would obscure the 2025–2026 frontier.
The following were not promoted to separate chapters in this batch:
- long video, because continuous single-shot generation and cross-shot narrative already have separate causal-streaming and story/multi-shot routes;
- video-to-audio, because it is used as the staged counterfactual inside the native AV chapter rather than mislabeled as joint generation;
- ~~multi-view and 4D generation were initially deferred because this batch treated them as geometry-heavy branches of explicit controllability.~~ A follow-up audit on 2026-08-30 promoted them to their own chapter: camera-controlled diagonals, same-time multi-view grids, renderable dynamic states, unseen-region generation and 2026 long/streaming 4D work now require a distinct task and evaluation contract. The follow-up evidence is recorded in the dedicated research log.
- generic safety and provenance, because they already have explicit sections in evaluation, applications, datasets and foundation-model release-surface audits.
4. Independent primary-source frontier check¶
The purpose of this table is to verify that each proposed chapter is anchored in a current technical transition, not only in an old taxonomy label. Results remain author-reported unless an independent reproduction is explicitly cited.
| Axis | Primary source | What it changes | Evidence status at freeze date |
|---|---|---|---|
| Native AV | NAVA | Separates native AV alignment from external context conditioning with Align-then-Fuse MMDiT | 2026 preprint |
| Native AV | Inference-Time Scaling for Joint Audio-Video Generation | Uses multiple verifiers and adaptive reward weighting; explicitly reports single-verifier trade-offs and hacking risk | TMLR acceptance stated on arXiv; final journal metadata must still be version-checked |
| Native AV | Ripple | Adds modality-specific recurrent memories, cross-modal interaction and block-causal streaming | 2026 preprint; latency/FPS are author protocol results |
| Post-training | Improving Video Generation with Human Feedback | Jointly exposes multi-dimensional VideoReward, Flow-DPO, Flow-RWR and inference-time Flow-NRG | NeurIPS 2025 formal publication |
| Post-training | Dual-IPO | Iteratively updates both reward model and video generator instead of freezing one side | ICLR 2026 formal publication |
| Post-training | Consistent Noisy Latent Rewards | Moves reward evaluation into noisy latent trajectories and tests preference consistency across timesteps | ICLR 2026 formal publication |
| Post-training | BranchGRPO | Amortizes shared rollout prefixes and assigns depth-wise advantages instead of broadcasting one terminal reward uniformly | ICLR 2026 formal publication; video evidence is one reported setting |
| Post-training | DynamicsBoost | Builds preference order from continuation context and excludes the shared prefix from the DPO loss | CVPR 2026 formal publication |
| Controllability | Motion Prompting | Treats sparse or dense motion trajectories as a general motion prompt | CVPR 2025 formal publication |
| Controllability | GEN3C | Renders an incrementally updated 3D cache under a target camera path before conditioning generation | CVPR 2025 formal publication |
| Controllability | LAMP | Translates cinematography language into deterministic object/camera trajectory programs | CVPR 2026 formal publication |
| Controllability | BulletTime | Explicitly decouples world time from camera pose | CVPR 2026 formal publication |
| Controllability | FlashMotion | Shows that distilling a base generator does not preserve a control adapter automatically and adds post-distillation adapter tuning | CVPR 2026 formal publication |
| Controllability | 4DStreamCtrl | Combines camera and object control with online streaming and long rollout | 2026-08-26 preprint; real-time and long-run numbers are author-reported |
5. Integration contract¶
The three chapters must remain connected but not collapsed:
- Post-training may optimize AV synchronization or trajectory adherence, but the reward does not redefine the underlying task.
- Few-step distillation changes the sampling budget; it is not preference alignment even when both appear in one recipe.
- Camera and trajectory conditions specify a desired visual motion; they are not environment actions unless the system establishes a state transition and closes the action-observation loop.
- A video with sound is not native joint AV unless the public mechanism shows audio and video exchange information during generation.
- Products, papers, code, weights and current endpoints are separate release surfaces and receive separate labels.
The repository integration therefore requires links from the task taxonomy, mechanism overview, foundation-model overview, reading route and top-level README, plus a dated update to the coverage matrix. It does not require forcing every chapter-local paper into the core bibliography registry.
6. Evidence boundary¶
This batch is a literature, artifact and documentation audit. It does not independently train or reproduce the cited large models. Mermaid rendering, image inspection, link closure and metadata checks can prove that the educational artifacts are internally sound; they cannot convert author-reported quality, latency, synchronization or controllability into independently verified performance.