Use Remotion to create great video ads
Remotion renders video from React, which makes it a good fit for ad work where you need the same thing forty times with one variable changed. Scope: one English-speaking founder on camera, phone footage, no B-roll, no music. 48 ads from one 24-minute shoot.
To generate video ads with Remotion you write the ad as a React component, drive every animation from useCurrentFrame(), and render the same composition into each aspect ratio you need. That part is straightforward and comes last. The work that decides whether the ads are usable happens before it, in this order: cut the raw footage where the speaker has actually finished a thought, get the loudness and the room out of the audio, burn in captions that are spelled correctly and land on the right frame, then compose, then check. This guide follows that order.
We built 48 ads out of 24 minutes of phone footage this way. Most of what follows is what went wrong the first time, because that turned out to be the useful part. Versions, so the rules below are true for what you install: Remotion 4.0.499 with @remotion/media, whisper.cpp 1.5.5 with the medium.en model, Python 3 with nara_wpe, ffmpeg 7.
The short version
- Remotion is one of 28 scripts in the pipeline. It renders the ad. Deciding where to cut, fixing the audio and checking the output are the rest, and they are where the quality lives.
- Cut on the voice, not on the transcript. A transcript knows where a sentence ends as text. It cannot tell you whether the speaker had finished, and cutting a beat early is the thing that makes an ad sound amateur.
- Never normalise with single-pass
loudnorm. It is a dynamic filter: it returned clips 360ms short and compressed away the level fall that makes an ending sound final. - Fix caption text at the sentence level, not the token level. Whisper splits words unpredictably, so
"Adside.ai"can arrive as four tokens and a token-level rule silently never fires. - Measure the artifact, never the intent. Every expensive mistake came from trusting what a step meant to do rather than checking what it produced.
When code beats an editor
Generating video with code is worth it when you need many variations of a structured video: one ad in three aspect ratios, twelve hooks over one body, forty cuts of a single interview. The win is repeatability. Change the end card once and 48 ads update.
It is not worth it for a one-off brand film. An editor will do that better and faster, and nothing about the approach helps you.
The case for volume is made in how many creatives you actually need and the testing framework; this guide assumes you've accepted it.
One thing to settle before you build anything: Remotion is free for individuals, non-profits and for-profit companies with up to 3 employees. For-profit companies with 4 or more employees need a paid Company License.¹ The licence is per legal entity: an agency with four people needs it whatever the client's size. Worth knowing on day one rather than the day the pipeline becomes load-bearing.
Cut on the voice, not the transcript
This is the part that decides whether the ads feel professional. We got it wrong first.
A transcript tells you where a sentence ends as text. It cannot tell you whether the speaker had finished. English statements close on a terminal fall: pitch declines across the last syllables, the level tails off, and a pause follows. Cut before that lands and the ad sounds interrupted even though every word is present.
So measure three things at any candidate cut point, from the audio itself:
| Signal | What it tells you | Threshold we use |
|---|---|---|
| Pause after the cut | Whether the speaker stopped at all | ≥0.3s; ≥0.7s settles the acoustic test on its own |
| Pitch fall | Whether the sentence closed or is still running | Closing F0 ≤93% of the utterance's own median |
| Level decay | Whether the delivery tailed off | ≥3dB drop into the cut |
Below a 0.7s pause, require a pause plus either the pitch fall or the decay. Two implementation traps cost us most of a day each:
If your probe point lands inside a speech region, do not measure the pause to the next region. That counts the rest of the sentence the speaker is still saying as silence, and a mid-word cut scores as a 1.9-second pause. Return zero instead.
And autocorrelation pitch detection reads an octave low more often than you would expect. Everything measured a flat 82Hz until we added octave correction; the real voice was 110–140Hz falling to 83Hz at a terminal fall. Without that correction every ending looks equally flat, so the whole check silently does nothing. These numbers are one male voice; the 0.93 ratio is relative to the utterance's own median so it should transfer, but re-tune it on your speaker before trusting it.
Cadence proves the speaker stopped. It cannot prove the thought was complete.
A half-second breath mid-sentence passes every acoustic test, which is how we shipped an ad ending "…paired with a human expert we managed." The fix is to also require the transcript to end on terminal punctuation, or at least not on a word that cannot end a clause.
Trim the start too. Leading silence is dead air at the top of the ad, and it also makes Whisper smear the first words backwards across it: 1.7 seconds of room tone made it place "I believe that" at 0.00s when the speaker doesn't start until 1.71s. Beginning the cut about 120ms before the first word fixes the caption timing and the dead air together.
Finally, accept that some takes cannot be saved. Three of our recordings had no point at which the voice landed, because the camera was stopped while the speaker was still talking. That is a shooting problem, and no amount of editing fixes it. A beat of silence before you stop recording is the cheapest thing anyone can do to make footage editable.
Take the dead air out
Around 17% of our raw runtime was not speech: pauses between sentences, all of which were sitting in the ads. Cutting every gap over 0.26s down to 0.1s made them noticeably punchier without touching a word. On one clip that was 20.5s down to 16.6s.
Removing a pause from a locked-off talking head leaves a jump cut, which reads as a dropped frame. Change the shot size across the cut instead: step the framing between wide and about 6% punched in, instantly rather than eased. That turns each removed pause into a visible edit.
The clips are cut and tightened. Next, the audio inside them.
Audio is where ads get thrown away
Do not normalise with single-pass loudnorm. It is a dynamic filter, and two things go wrong that are invisible unless you measure the delivered file. Its lookahead is not flushed, so clips come out short: ours lost 360ms, which was the end of the last word, while the sidecar (the per-clip JSON the cut script writes) still recorded the duration that had been requested. Every downstream check therefore agreed the cut was clean. And being dynamic, it compresses, lifting the quiet tail of each sentence back toward speaking level and erasing the falling loudness that tells a listener a thought has finished.
Instead, measure integrated loudness with ebur128, compute a fixed gain, and apply it linearly. EBU R128 is the broadcast loudness standard the tooling implements.² Meta publishes no loudness target; −16 LUFS integrated is the de facto number streaming platforms normalise toward, so it's our house target, with the QA band at −17 to −11.
Two gotchas around that. FFmpeg writes measurements to stderr, not stdout: capture only stdout and you silently get your fallback value, which in our case produced an "+83dB correction". And sample peak is not true peak: limiters clamp the former, delivery specs are written in the latter, and the reconstructed waveform overshoots between samples. Limiting to exactly −1.5 delivered −0.7 dBTP, so give the limiter about a decibel of extra headroom.
Reverb: the intuitive tools are the wrong ones
Our footage was shot outdoors and in a live room, and the echo was the most common note we got. The obvious moves are a spectral denoiser or a downward expander, and both are wrong. A denoiser assumes the unwanted part is stationary, but reverb is a delayed copy of the voice itself, so it removes the wrong thing and leaves watery artefacts. An expander only ducks the tail between words, leaving the room under every syllable.
An expander steepens the level decay by construction, which is exactly what an RT60 estimate measures. It scored beautifully on our metric while doing almost nothing audible: one clip measured 0.25s under the expander and 0.70s without it. If your processing can flatter your measurement directly, the measurement is not evidence.
The right tool is real dereverberation. WPE predicts late reverberation from the signal's own recent past and subtracts it, so what is removed is by construction a delayed copy of the voice.³ It's a Python library (nara_wpe) in a Node pipeline: we pipe raw float PCM from ffmpeg into a 40-line Python script and back. That took RT60 from 0.70s to 0.30s with the transcript word-identical. Afterwards, keep it minimal: a small dip around 300Hz if the room is boxy, a little presence around 4kHz, gentle levelling.
Do not add a high-pass afterwards. A 2-pole IIR rings at its corner frequency, and that ringing is itself a decaying low-frequency tail: it put RT60 back to 0.35–0.40s, undoing part of the fix.
Clean, level audio. Now the words on screen.
Captions, spelled correctly
Feed video plays muted by default, so burned-in captions are not optional. Transcribe locally with whisper.cpp⁴ (we use @remotion/install-whisper-cpp, which installs it and fetches the model; it needs 16 kHz mono WAV in, so there is an ffmpeg -ar 16000 -ac 1 step first, and token-level timestamps on). Use medium.en for anything burned into a frame; base.en is where "paid ads" becomes "Paydads".
Corrections have to be durable. Whisper repeats the same mistakes on the same voice, and fixing the caption file by hand works exactly until the clip is re-cut, at which point it is re-transcribed and every edit is silently lost. That happened to us three times before we moved corrections into clients/<slug>/caption-fixes.json, two lists (words for single-token swaps, phrases for errors that span tokens) re-applied automatically after every transcription.
Fix at the text level, not the token level. Whisper splits words across tokens unpredictably and differently every time:
A rule written as ["@","site"] never fires if your normaliser strips @ to an empty string, which ours did. Match against the joined sentence and map the match back onto whichever tokens cover it, and tokenisation stops mattering. Watch two details: a multi-word replacement fuses into paidads unless you re-introduce the separator, and a denylist must never match the correct spelling. Ours flagged "Adside" itself, which made every clean caption a violation.
On timing, the target is that a caption page appears within 0 to +220ms of the words. Slightly late is invisible; early means the viewer reads the line and then waits for it. Snap per token against the audio rather than per utterance, because the gaps between tokens are exactly what Whisper gets wrong: on one take the speaker says "We're", pauses 1.6 seconds, then "building", and Whisper had stretched "'re" across the whole pause so there was no gap in caption time at all.
Finally, size captions by measurement rather than authoring them. Measure the widest unbreakable word, cap the font size so it fits the column, count the wrapped lines and shrink until the block fits the height. One CSS trap: white-space: pre preserves the leading space but forbids line breaks, so if every break opportunity sits inside such a span, wrapping can never happen and the text overflows however carefully you computed the line count.
You now have cut clips, clean audio and corrected captions. Remotion turns them into ads.
Compose the ad in Remotion
A composition is a React component plus a frame rate, dimensions and a duration. useCurrentFrame() tells the component which frame it is drawing, and the renderer walks the frames and encodes them.⁵
The structural decision that matters is keeping content out of components. An ad is a data file; the template is code. Ours has five templates (hook-proof-cta, ugc-testimonial, problem-solution, product-demo, stat-listicle) and every one of the 48 ads is a file like this one, which picks a template, a clip and the formats to render:
So an ad here is one clean segment of the founder talking (2.8 to 54 seconds across the 48, most around 20), burned-in captions, and a 2.8-second branded end card, rendered at 30 fps H.264 into out/<client>/<ad-id>/<ad-id>--<format>.mp4. Brand fonts load through @remotion/google-fonts for Google fonts and @remotion/fonts for local files; colors and the logo live in the client's brand.ts. When those mix into the template, ad number thirty is a copy-paste of ad twenty-nine and nothing can be changed globally.
Four rules that are specific to Remotion and cost us time:
- CSS transitions and animations are nondeterministic. They run on wall-clock time while the renderer screenshots frames, so you get whatever state the browser is in when the frame fires. Drive every animation from the frame number.
@remotion/media's<Video>draws to a canvas, soobjectFitis a prop, not a style. Set it instyleand your footage letterboxes silently. There is noobjectPositioneither; re-crop with ffmpeg.- Round computed pixel values. A sub-pixel
gapcan serialise into exponential notation, which the style parser drops. Ours became zero and the words touched. - Use
<Video>and<Audio>from@remotion/media. Plain HTML tags do not sync to the frame timeline.
One ad, every format
Every ad declares which formats it renders, and layout code never touches a pixel value: sizes come from a scale unit derived from the canvas, positions from the safe insets in the registry. The registry is five entries. Adding a format is one more.
| Format | Canvas | Placements | Safe insets (top / bottom / sides) |
|---|---|---|---|
| reels | 1080 × 1920, 30 fps | Instagram and Facebook Reels, TikTok, Shorts | 14% / 35% / 6% (Meta official) |
| story | 1080 × 1920, 30 fps | Instagram and Facebook Stories | 14% / 20% / 6% (Meta's older Stories figure; use reels if the asset also runs there) |
| feed | 1080 × 1350, 30 fps | Facebook and Instagram Feed | 6% / 10% / 5% (house rule) |
| square | 1080 × 1080, 30 fps | Feed fallback, Marketplace, Audience Network, LinkedIn | 6% / 10% / 5% (house rule) |
| landscape | 1920 × 1080, 30 fps | In-stream, YouTube, right column | 6% / 12% / 6% (house rule) |
Delivery is H.264 in MP4 at 30 fps with AAC audio, which Ads Manager takes without re-encoding; Meta accepts up to 4 GB and recommends 1440 × 2560 for 9:16 and 1440 × 1800 for 4:5, so raise the canvas if you want the larger files.⁶ The 9:16 numbers are the same ones our static gallery draws; the full write-up with the pixel table is in the Claude Design guide, and the registry as JSON is in both runbooks. Filenames carry the ad id and the format (ycad-0007-1-learn-highly-targeted--reels.mp4): a number, then the words that distinguish this ad from the others, never the words they all share.
Everything up to here produces files. The last step is refusing to ship the wrong ones.
Checks that stop bad ads shipping
Every rule above exists because something shipped wrong. The only way they hold at volume is as checks that block the render, and the only way to trust a check is to deliberately break it once and confirm it fires.
| When | Check | Why it exists |
|---|---|---|
| Before render | Clip does not end while the voice is going | Measured on the file that will render, not on what the cut intended |
| Before render | Transcript ends on a finished thought | Cadence alone passes a mid-sentence breath |
| Before render | No caption more than 220ms early | Reading a line then waiting for it is very visible |
| Before render | No wrong brand spelling in any caption | Fixes run at transcription time and can be raced by a re-cut |
| After render | Integrated loudness inside −17 to −11 LUFS, true peak under −1 dBTP, consistent across the set | In a review session the ads play back to back; a 15 LU step is worse than either level |
| After render | No render older than its footage | Re-cutting a clip does not re-render the ads that use it |
A stale render is indistinguishable from a current one, so it cost us two review rounds where the reviewer reported bugs that were already fixed on disk. Comparing each render's timestamp against every asset it depends on, and marking the ones that are behind, removed the whole class of problem.
One design note: preflight only the ads you are about to render. Checking the whole project means a single unrelated problem blocks every render, which happened to us three times in one afternoon before we noticed the pattern.
If you take one thing from this: measure the artifact, never the intent. The truncated clips and the misspelled brand both shipped because a step recorded what it meant to do and nothing checked what it actually produced.
FAQ
Is Remotion free to use for commercial video ads?
It's free for individuals, non-profits, and for-profit companies with up to 3 employees, and those users can create commercial videos with it. For-profit companies with 4 or more employees need a paid Company License. The licence is per legal entity, so an agency with four people needs it whatever the client's size. Check the threshold before you build on it.
Can Remotion generate video ads automatically from footage?
Remotion renders the video, but it does not decide where to cut your footage or what to say. A working ad pipeline wraps it in three other stages: cutting clips out of raw takes at boundaries where the voice actually lands, transcribing and correcting captions, and checking the output before it ships. Remotion is one of 28 scripts in our pipeline, and the easiest one.
How do you add burned-in subtitles to a video in Remotion?
Transcribe the audio to word-level timestamps, usually with a local Whisper build, then render each caption page as a component driven by useCurrentFrame(). Two things matter more than the styling: keep a durable list of corrections that is re-applied after every transcription, because re-cutting a clip re-transcribes it and silently loses hand edits, and check that no caption appears more than about 220ms before the words are spoken.
Should I use Remotion or FFmpeg for generating video ads?
Both, for different jobs. FFmpeg is the right tool for cutting, trimming, loudness normalisation and any per-sample audio work. Remotion is the right tool for the visual layer: captions, motion, brand elements, end cards, and rendering the same ad into several aspect ratios. A practical pipeline shells out to FFmpeg to prepare clips and uses Remotion to compose the ad.
Why do my generated video ads sound like they cut off mid-sentence?
Because the cut point came from the transcript rather than the audio. A transcript tells you where a sentence ends as text, not whether the speaker had finished. English statements close on a terminal fall: pitch declines, level tails off, and a pause follows. If you cut before that lands, the ad sounds interrupted even though every word is present. Measure the pause, the pitch fall and the level decay at the cut point, and require a real pause of at least 0.3 seconds.
How much of a raw interview ends up usable as ad footage?
Less than people expect, and the limiting factor is clean endings rather than content. Across 36 takes we found 77 usable segments, and three whole recordings had no point at which the voice landed, because the camera was stopped while the speaker was still talking. Around 17% of the runtime that did work was silence between sentences.
Sources
- Free licence eligibility and the 3-employee threshold: Remotion License, GitHub
- Loudness normalisation standard used by the measurement tooling: EBU R 128, European Broadcasting Union
- Weighted prediction error dereverberation: nara_wpe, Paderborn University
- Local word-level transcription: whisper.cpp
- Compositions, the frame timeline and rendering: Remotion documentation
- Reels video specs, 1440 × 2560, MP4/MOV, 4 GB: Instagram Reels video ads, Meta Ads Guide