HyperFrames by HeyGen: Video as Code for AI Agents and Codex

HyperFrames by HeyGen is an open-source framework where video is assembled from HTML, CSS, and JavaScript by an AI agent's own hands. Here's a look at its capabilities, use cases, and how it integrates with Codex.

When to use it. HyperFrames earns its place whenever video needs to be built with code and an agent instead of a timeline editor: product videos, breakdowns, social clips, data animations, or videos generated from documentation. For a plain landing page, you don't need it — this is a tool for reproducible video.

What Is HyperFrames

HyperFrames solves one specific problem: giving an AI agent a way to make video in the language it understands best. Coding agents aren't great at clicking around in Premiere or After Effects, but they're excellent at writing markup and scripts. HyperFrames turns video production into code, so the agent works with a video the way it works with any other project.

Key facts:

• Open source, Apache 2.0 license.
• The principle is "one input file, one video output."
• Rendering is deterministic: the same composition always produces the same result, frame for frame.
• Works locally via CLI, from coding agents through skills, or as a rendering engine inside a larger pipeline.

This isn't about avatars. Classic HeyGen turns text into talking heads; HyperFrames assembles the composition of the video itself — titles, animation, captions, graphics, transitions.

How It Works: Video as Code

A composition is an HTML document. Timing lives right in the markup, in data attributes. GSAP and other runtime adapters handle the animation, and FFmpeg assembles the final MP4.

A minimal composition looks like this:

<div id="root" data-composition-id="my-video"
     data-start="0" data-width="1920" data-height="1080">
  <h1 id="title" class="clip"
      data-start="0" data-duration="5" data-track-index="0">
    Hello, HyperFrames!
  </h1>

  <script src="https://cdn.jsdelivr.net/npm/gsap@3/dist/gsap.min.js"></script>
  <script>
    const tl = gsap.timeline({ paused: true });
    tl.from("#title", { opacity: 0, y: -50, duration: 1 }, 0);
    window.__timelines = window.__timelines || {};
    window.__timelines["my-video"] = tl;
  </script>
</div>

Three rules keep a composition working:

1. The root element sets data-composition-id, data-width, and data-height.
2. Every timed element gets the clip class plus data-start, data-duration, and data-track-index attributes.
3. The GSAP timeline is created with { paused: true } and registered on window.__timelines.

You don't have to write this by hand — that's exactly what skills are for, teaching the agent the correct syntax. More on that below.

What You Get: Capabilities

Capability	What it gives you
HTML/CSS/JS compositions	Build video like a web page, reuse components
Animation runtime adapters	GSAP, Lottie, Three.js, Anime.js, CSS, WAAPI, TypeGPU in one composition
Catalog of ready-made blocks (50+)	Transitions, social overlays, charts, effects with one command
Website capture	Grabs screenshots, tokens, fonts, and assets from a site to build a composition
Captions and voiceover	TTS, transcription, kinetic subtitles
Deterministic rendering	The same result on every run — handy for pipelines

Installation: Two Paths

Path 1 — Through an AI Agent (Recommended)

Install the skills, then simply describe the video you want in plain language:

npx skills add heygen-com/hyperframes

The installer will show a list of skills. Grab the core set that any project needs (the --all flag installs everything at once). Skills register as slash commands in Codex, Claude Code, Cursor, Gemini CLI, and GitHub Copilot CLI.

Path 2 — Manually via CLI

npx hyperframes init my-video
cd my-video
npx hyperframes preview

init scaffolds the project structure and installs the agent skills automatically, so you can hand the work off to an agent at any step. preview opens HyperFrames Studio in the browser with hot reload.

Final render:

npx hyperframes render --output output.mp4

Prerequisites

• Node.js 22+ — the environment for the CLI and dev server.
• FFmpeg — encodes the local render.
• Docker — optional, for a fully reproducible render.

Integration with Codex

In Codex, HyperFrames connects as a plugin, and its skills become slash commands. From there, you work with video in the same window where you build sites and edit infrastructure.

Setting It Up in Codex

The main path: Plugins → HyperFrames by HeyGen → Install. An alternative is to install the skills from the upstream repository:

npx skills add heygen-com/hyperframes --full-depth

For local preview and rendering, Node.js 22+ and FFmpeg need to be present in the environment.

Slash Commands

The core set worth installing in full:

Command	What it loads
/hyperframes	Entry point, read first: orients and routes the request to the right workflow
/hyperframes-core	Composition contract: HTML structure, data attributes, clips, tracks
/hyperframes-animation	Animation and runtime adapters (GSAP, Lottie, Three.js, Anime.js, CSS, WAAPI, TypeGPU)
/hyperframes-creative	Art direction: palettes, typography, voiceover, storyboarding
/hyperframes-cli	Dev loop: init, lint, validate, preview, render, doctor
/media-use	Asset prep: TTS, transcription, background removal
/hyperframes-registry	Installing blocks and components from the catalog
/general-video	General workflow, a fallback for any video

Workflow skills are installed as needed: /product-launch-video, /faceless-explainer, /pr-to-video, /embedded-captions, /talking-head-recut, /motion-graphics, /music-to-video, /slideshow, /remotion-to-hyperframes.

Start your prompt with /hyperframes — it will route the task to the right skill on its own.

Don't skip the slash command. Without /hyperframes, the agent starts guessing at the rules of HTML video and breaks the composition — skipping class="clip" or forgetting to register window.__timelines. The slash command loads the framework's actual rules and gets you a correct result on the first try.

Cold and Warm Starts

Cold start — describe the video from scratch:

Using /hyperframes, build a 10-second intro video: a title with fade-in
on a dark background and quiet background music.

Warm start — turn material you already have into video. For Codex, this is the strongest scenario, because the agent already has the context of the repo, the PR, and the docs:

Look at this repository and explain its architecture with a 45-second video
through /hyperframes.

Build an animated bar chart race from this CSV through /hyperframes.

Then you iterate like you would with a video editor, in plain text:

Make the title twice as big, switch to a dark theme, and add a fade-out at the end.
Add a lower third at 0:03 with my name and title.

Smoke Test in Codex

npx hyperframes doctor

If the environment is ready:

npx hyperframes init smoke-video
cd smoke-video
npx hyperframes preview

The smoke test passes if doctor finds Node and FFmpeg, and preview opens without needing cloud credentials.

Setting Boundaries via AGENTS.md

The CLI writes project files, runs Node, does headless rendering, and calls FFmpeg. Local rendering doesn't require the cloud, but TTS, hosted rendering, and external media may need credits. To keep the agent from wandering into publishing or cloud usage without asking, set boundaries in AGENTS.md: allow local preview and render, forbid cloud rendering and any outbound sending without separate confirmation.

Tip: HyperFrames pairs well with other Codex skills: a product design skill sets the direction, GSAP handles meaningful motion, and a QA-focused skill checks quality. In that chain, HyperFrames is responsible for the final video.

Key CLI Commands

Command	Purpose
npx skills add heygen-com/hyperframes	Install the skills (--all installs everything)
npx hyperframes init <name>	Scaffold a project (supports --non-interactive, --example)
npx hyperframes preview	Browser preview with hot reload
npx hyperframes render --output out.mp4	Final render to MP4
npx hyperframes add <block>	Add a block from the catalog (captions, data-chart)
npx hyperframes capture <url>	Capture a website into frames
npx hyperframes lint / check	lint checks structure; check does a full render check: runtime, layout, motion, contrast
npx hyperframes doctor	Diagnose the environment (Node, FFmpeg)
npx hyperframes@latest upgrade	Update the project and skills

Skills and Ready-Made Blocks

Skills encode what standard web documentation doesn't cover: the mandatory class="clip" on timed elements, GSAP timeline registration, the semantics of data attributes, adapters like window.__hfLottie. That's why an agent with skills produces a correct composition right away, on the first pass.

The block catalog offers 50+ ready-made elements: transitions, social overlays, data visualizations, effects. They're added with a single command:

npx hyperframes add instagram-follow
npx hyperframes add data-chart

Use Cases

Task	Skill or approach	Result
Product announcement	/product-launch-video	A feature video with brand elements and storyboarding
Product tour from a site	capture • /general-video	Screenshots and site assets assembled into a tour with captions and voiceover
PR breakdown for reviewers	/pr-to-video	A video briefing on the diff without opening an editor
Faceless explainer	/faceless-explainer	Script, voiceover, motion graphics, subtitles
Data animation	add data-chart	A bar chart race from CSV
Social clip with a hook	/motion-graphics, /embedded-captions	Vertical video with kinetic subtitles
Talking-head video with graphics	/talking-head-recut	A finished interview with lower thirds, titles, and data callouts

Below are three scenarios that map closest to working inside Codex.

A Video from a Repo or Docs

Codex already has access to your code. Give it the task right inside the project's context:

Using /hyperframes, build a 60-second video explaining what this service does.
Pull the architecture from the README and the diagram from docs.
Vertical format 9:16, kinetic subtitles synced to TTS voiceover.
Local preview only, no cloud.

A Video Tour of a Website

Capture https://example.com using npx hyperframes capture,
then build a 30-second product tour from the captured material with captions
for three screens and light transitions. Use /general-video.

Data Animation from a Spreadsheet

Take data/metrics.csv and build a 15-second animated bar chart race
using /hyperframes with the data-chart block. Label the axes and add background music.

A general technique for all three: first ask the agent to show a composition plan (scenes, duration, assets), approve it, and only then run preview and render. That way you're revising a cheap plan instead of an expensive render.

Limitations and What to Plan For

• Entry barrier. You need a coding agent or CLI and a basic understanding of HTML. This isn't a mouse-driven GUI editor.
• Cost. The framework itself is free and open, and local rendering costs nothing. Cloud rendering, TTS, and external media may require credits and a HeyGen account.
• Determinism. Compositions need to be predictable. Infinite loops and unsynced timelines get in the way of rendering.
• Quality vs. speed. High settings noticeably slow down the build. Balance them against the task at hand.
• Diagnostics. If something breaks, lint, check, and doctor catch structural and runtime errors, missing assets, and contrast issues.

The trade-off is an honest one: you get reproducible video and full control in exchange for building it with code. For a one-off greeting card, a regular editor is simpler; for a steady stream of videos out of a single pipeline, HyperFrames wins.

Quick Checklist

• Node.js 22+ and FFmpeg installed (npx hyperframes doctor)
• Skills installed: npx skills add heygen-com/hyperframes
• The HyperFrames by HeyGen plugin is connected in Codex
• Project scaffolded: npx hyperframes init
• The prompt starts with the /hyperframes slash command
• The composition passes lint and check
• preview opens locally without cloud credentials
• Boundaries are set in AGENTS.md: local rendering is fine, cloud and publishing require confirmation
• The final MP4 is produced via render

Reference Links

• Site and documentation: hyperframes.heygen.com
• Quickstart: hyperframes.heygen.com/quickstart
• Prompting guide: hyperframes.heygen.com/guides/prompting
• Block catalog: hyperframes.heygen.com/catalog
• Source code: github.com/heygen-com/hyperframes
• HeyGen help article: HyperFrames x HeyGen

If you're already building sites and content through Codex and want to add reproducible video from the same pipeline, HyperFrames fills exactly that gap — turning video production into another kind of code your agent already knows how to write.