Open-source video production benchmarks

Updated: 2026-09-01

Conclusion

The previous AI Video Lab workflow was too narrow. It compared generation models and hand-built Remotion scenes without first benchmarking mature agentic production systems. OpenMontage is now the primary architecture benchmark, ViMax is the secondary orchestration/consistency benchmark, and editing-automation projects are candidates for the finishing layer.

OpenMontage

Source: https://github.com/calesthio/OpenMontage

Adopt as design requirements:

• Reference-video-first analysis: transcript, pacing, scenes, keyframes, style, what to keep/change, cost estimate, and sample before full production.
• Pipeline selection before tool selection: research -> proposal -> script -> scene plan -> assets -> edit -> compose.
• Scene-by-scene contact-sheet approval with takes, prompts, cost, and quality scores.
• Real-motion retrieval as a first-class path, instead of treating animated stills as a finished video.
• Provider decision logs, delivery-promise validation, ffprobe/frame/audio/subtitle QA.
• A live production board derived from project artifacts rather than a manually maintained marketing page.

Do not copy implementation code into the proprietary AVS system without a license decision. OpenMontage is AGPL-3.0; use its public architecture and workflow as a benchmark unless reuse is separately approved.

ViMax

Source: https://github.com/HKUDS/ViMax

Adopt as design requirements:

• Explicit Director / Screenwriter / Producer / Generator responsibilities.
• Script2Video as the default for controllable brand films; one-prompt Idea2Video is only for exploration.
• Reference, first-frame, character, object, location, and camera-continuity tracking across shots.
• Persistent sessions, artifact/storyboard/render previews, checkpoints, and resumable generation.
• Parallel shot generation only after references and continuity constraints are locked.

ViMax is MIT-licensed and is a stronger candidate for selective implementation reuse, but its quality claims still require local evaluation on PENGIN assets.

video-editing-automation topic

Source: https://github.com/topics/video-editing-automation

Evaluate separately:

• premiere-pro-mcp: finishing, timeline editing, audio, captions, and human handoff in Premiere.
• stockpile: semantic B-roll retrieval pattern, adapted to PENGIN/Drive/local assets instead of generic stock.
• capcut-mcp-full: possible short-form delivery path, but only after project-file integrity and Windows operation are tested.

The topic page itself is discovery input, not a quality benchmark. Repository activity, licenses, actual output, and project-file reliability must be checked individually.

Immediate AVS changes

1. Reference video analysis becomes mandatory before story development.
2. Every scene gets a role, reference, source asset, prompt, provider, cost, continuity constraints, and acceptance score.
3. Full renders are blocked until the scene contact sheet passes.
4. Real footage and internal source assets are searched before generative footage.
5. The review page becomes a live production board with rejected takes and decision history.
6. The current 67-point film remains a rejected baseline, not an active final candidate.