An agent that films the real product and cuts it into narrated motion-graphics ads, and records proof videos for other coding agents

Problem

The manufacturing SaaS platform I work on keeps shipping features, and each one deserves a short ad: something a manufacturer can watch on a phone and understand quickly. Made by hand, every ad costs someone's design and editing time. Generated by a video model, it shows a product that does not exist.

Ad Studio makes product ads from the real thing: kinetic type, the actual UI in device frames, spotlights, a scripted cursor, stat counters, beat-synced cuts, a voiceover, a music bed, and a logo end card. The same captures produce stills for release posts and social cards. One rule is not negotiable: every frame of product is a real capture. No AI-generated UI, imagery or b-roll.

Constraints

• Look, never touch. The agent browses a live app to find what to film. It must not create, change or delete anything.
• No passwords. The person logs in themselves; the tool never handles a credential.
• Open source only. That ruled out Remotion (paid license), GSAP (not OSI-approved), and the MusicGen, XTTS and F5 weights (non-commercial). It also ruled out Playwright MCP, for a different reason, below.
• No API keys. All model work runs through the person's own Claude Code login.

Architecture

Web app

React, job queue, SSE

/ad skill

in Claude Code

Coding agents

verify MCP

→↓

Studio agent

no shell, per-job MCP

→↓

Guarded capture

Playwright, GET only

→↓

Storyboard

Zod contract

Brand kit

per-brand data

→↓

Render

HTML to frames

Voice and music

Kokoro, whisper

ffmpeg QA

LUFS, contact sheet

A local Bun server runs a FIFO job queue and streams progress over server-sent events to a React app. Each job starts Claude Code headless: restricted, with only Read, Glob and Grep, no session persistence, an empty temporary working directory, and its own HTTP MCP server reachable with a bearer token passed in a file, never on the command line. The agent's tools include look, goto, click, still, code_history, note, ask and propose_script. The same tool registry backs a stdio MCP server for the /ad skill, so the two surfaces cannot drift apart.

The pipeline.

1. Discover. The agent reads the feature's code and git history, then browses the app in look-only mode.
2. Ask. Questions appear as cards in the UI. Before opening anything sensitive, it must ask.
3. Script. It proposes a shot script. A person edits and approves it.
4. Record. The approved script compiles to a recipe that replays deterministically: 3x stills, 1x clips, and element boxes for spotlights.
5. Storyboard. Scenes are drafted from the script and validated against a Zod schema, including the brand's copy rules.
6. Build. Voice first, with word timings from faster-whisper; then music and a beat grid; then composition across seven scene templates; then the render. Full-quality output goes through a PNG sequence and one x264 encode, two-pass loudness normalization to -14 LUFS with a limiter, and a QA pass for duration, loudness, true peak and a contact sheet.

Stills take a shorter path: layout, static HTML, a Playwright screenshot, then QA for size, clipping, contrast and scale.

Key decisions

1. A network guard instead of trust. Only GET, HEAD and OPTIONS leave the capture browser. Exceptions are an allowlist of read-only POSTs and websockets, and each entry needs a logged block line as evidence that it was needed. Production hosts fail validation when the config loads.

2. No Playwright MCP. It can run arbitrary code in the page, and its per-origin filter is not a security boundary. A small set of purpose-built tools is easier to guard than a general browser.

3. The storyboard is data. Scenes are a Zod-validated document with fractional geometry, so a storyboard survives a re-capture at a different size. The renderer sits behind an interface, with Motion Canvas kept as a fallback. Brands are data too: kit, copy rules, sites and targets live in a brand folder, and nothing in the engine names a product.

4. Local and single-user. No hosting, no accounts, one browser per site at a time. Hosting was deliberately deferred, and it is the first thing to change if more people use it.

5. Open source has a price. The voice is good. The music is still a synthesized placeholder, because the open music model I would have used, MusicGen, ships non-commercial weights.

Proof videos for coding agents

The same capture stack became the verify stage of my agentic workflow. A coding agent calls verify_request and verify_wait. The queue hands it the shared local copy of the full stack one agent at a time, switches that environment to the ticket's worktrees, and has an agent film a narrated video of the change working.

Two tickets were filmed back to back with no manual step, about 15 and 11 minutes each including the switch. The first videos were not good enough, and I rejected them. The diagnosis found seven pipeline defects and four agent defects, among them a still taken before an in-app navigation finished, narration landing after the action it described, and focus boxes too large to point at anything. A rework followed.

What broke

• Every clip filmed the wrong corner. Emulating 3x scale while recording zoomed the page, so clips showed the top-left third and clicks landed in the wrong place. Clips now record at 1x; stills stay at 3x.
• Every film of the local stack was empty. The guard closed the dev server's hot-reload websocket, and without it the app made no API calls at all. A read-only socket allowlist fixed it.
• Questions died after 300 seconds. Claude Code aborts an MCP tool call that stays silent for 300 seconds regardless of the global timeout setting. A per-server timeout fixed it, proven by holding one question open for 15 minutes.
• The browser closed mid-question, because its idle timer kept counting while the agent waited on an answer.
• Dead sessions looked alive. After a failed refresh, the auth library keeps the user with an expiry in 1970. The session check now reads the expiry.
• Recipes broke between exploring and recording. Targets named by relative time, like "7m ago", change between runs. The recipe compiler now refuses them.
• Export quality. Frames were captured as JPEG. PNG frames plus one encode raised SSIM from 0.99351 to 0.99651 and cut the file from 16.0 to 10.8 MB.
• Dead time. ffmpeg's freeze detection now trims frozen stretches; one film went from 48.5 to 40.0 seconds.

Results

Built in about six days, 8 to 13 October 2026: 30 commits, about 16,700 lines of TypeScript, and about 214 test cases in 27 files. Output goes up to 4K vertical. QA holds every ad to -14 LUFS within 1.5, with true peak under -1 dBFS; the reference ad is 42 seconds at -14.1 LUFS.

It has produced close to ten product videos so far. The verification queue has run 11 requests: 6 completed, 4 failed and 1 cancelled.

What I'd change

• Real music. A licensed library, or a commercial-use model when one is good enough.
• Share it. Local and single-user was right for a first version. The next people to use it should not need my laptop.
• Make verification videos boring. Four failures in eleven is too many for a step agents rely on. Each failure class needs a check before filming starts, not a fix after.