Mass-producing music videos with MiniMax H3

The animated music videos on this site cost $0.29–$1.00 of GPU time per song. Sixty seconds, 10–23 cuts, and 23–41 minutes of generation. This article lays the machinery out at a level of detail that lets someone else walk the same path.

Not a soft overview: the models in use, the pinned commits, the actual commands and the measured numbers, as they are. Anything not written here is not being used.

One caveat up front. The mvpipe that appears below is a home-grown, private pipeline, not a package you can install. Typing the commands verbatim will not work. They are here to show concretely what each step takes as input and what it produces. The off-the-shelf parts are ComfyUI, rclone, ffmpeg and the RunPod API; what mvpipe does is call them, and manage the cut list, round to the frame lattice, verify, and assemble. Build the same structure and you get the same result. The step-by-step section is written so you can tell what needs building.

00The conclusion first — why "one cut at a time"

First, the fact that decided this whole design. H3 generation time degrades worse than quadratically with clip length.

A 3-second cut (73 frames) finishes in about 3 minutes. An 8-second cut (192 frames) does not finish in 75 minutes. It timed out three times in a row (the per-job limit is 4,500 seconds), and 14.4 seconds (345 frames) did the same. That experiment cost $3.18 and produced not one usable second of footage. Cutting the reference images from five to three changed nothing, so the hypothesis that the reference count was the cause is also rejected.

So "generate several cuts in one pass" does not work on the current stack. Generate short cuts one at a time and join them in the edit — that is the only practical answer. Everything that follows is derived from it.

01Cast and roles

One human, two kinds of AI, and a disposable GPU. Who holds what, first.

Codex CLI is used for image generation because it is $0 extra within the ChatGPT plan. It is launched as below; the default codex-family models are refused on a ChatGPT account, so the model is given explicitly. Reference images go in the current directory.

codex exec --sandbox workspace-write -c model="gpt-5.4"

Throughput is 8–10 minutes per image, sequentially. One song has been through 24 generated-and-approved images this way. That said, the mainstay today is the key visuals micha makes with GPT-Image2; Codex CLI inside the pipeline fills the gaps.

02Architecture

Five design principles. Not preferences — each was set by an accident or a bill.

1. Local is the source of truth, the GPU Pod is disposable, and all state lives on Google Drive. Build on the assumption that the Pod may die
2. Music-to-picture sync is done in the edit. The audio H3 outputs is discarded and the Suno master (48 kHz) is laid back in
3. Lip-sync (ref2va) is limited to singing cuts and the title drop. Everything else uses the faster fl2va
4. A key visual with lyrics burned in is used only where those lyrics are actually sung
5. A human approves before any GPU billing. Only after both the kv-review approval and the preflight approval does launch (= billing) begin

Four components: the local Python pipeline, the queue on Drive, the disposable Pod, and the Codex CLI key-visual line.

mvpipe is split into the units below. If you build your own, these fourteen are what you need.

Drive is the queue because state survives the Pod dying. Pod telemetry (CPU/GPU utilisation) is not trusted. On community Pods it can disagree with reality, so health is judged solely by whether files appear in results/ and failed/.

03Pinning the environment

This is the most important part for reproducing anything. Everything is pinned to a commit hash. Tooling around H3 moves fast, and without pins the same input stops giving the same output.

Weights come from Comfy-Org/MiniMax-H3 on HuggingFace at the revision above, via hf_hub_download. After downloading, the worker checks size and sha256. Five files are used.

Generation defaults are 480×864 (portrait 9:16), 24 fps, 20 steps, res_multistep/simple. Pod selection pins CUDA>=12.8 and RAM>=45GB.

H3 has two hard constraints. Both bite at design time.

• The frame count can only take values of the form 17k+5. Discrete values like 73f = 3.04 s, 158f = 6.58 s, 192f = 8.0 s. The training range is 124–362 frames. You cannot ask for an arbitrary number of seconds, so every cut length gets rounded onto this lattice
• Resolution must be a multiple of 32. Reference key visuals must be centre-cropped to the canvas ratio beforehand. fl2va applies a non-uniform stretch, so a mismatched ratio distorts the picture

In addition, audio passed to Ref2VA must be 15 seconds or less, and the per-job timeout is 4,500 seconds.

04Choosing the generation mode

There are three modes, chosen per cut. They differ threefold in speed, so this choice sets the cost.

ref2va gets the lip-sync right but is about three times slower than fl2va, so it is used only for cuts that are sung and the cut that lands the title. Made entirely in ref2va a song costs $1.00; mixed, it drops to $0.29–$0.51. That is where the price range at the top comes from.

The acceleration combination also changes by mode: Sage + Spectrum for ref2va, Sage + EasyCache for fl2va and t2va. An A/B test showed EasyCache actually slowed ref2va down (73 frames: 185.8 s → 195.5 s).

05Making one video — ten steps

Now the main part. Each step with its actual command, in order. <s> is the song slug; the working directory is pipeline/.

Analyse (step 2)

uv run python -m mvpipe.analyze ../songs/<s>/audio.mp3 \
  --lyrics ../songs/<s>/lyrics.txt --language ja --no-vad

Everything downstream stands on this. It does three things.

1. Beat detection. librosa reads the audio and beat.beat_track gives the BPM and beat positions. Downbeats (bar starts) are taken every four beats from the first beat. There is no time-signature analysis — a deliberate assumption of 4/4.

2. Lyric transcription. The faster-whisper medium model runs on CPU in int8 and yields word-level timestamps. This is where --no-vad is mandatory: with voice-activity detection on, a loud chorus gets classified as "not speech" and dropped wholesale. The thicker the backing, the more is lost, so it is off by default.

3. Matching against the lyric file. Whisper's transcript contains errors, so it cannot be used as-is. The correct lyrics on hand and the transcript are matched character by character (difflib.SequenceMatcher), and the timestamps of matched characters are transferred onto the correct lyrics. Unmatched characters are filled by linear interpolation within the line.

The advantage of this approach is that the string displayed is always the lyric on hand. Whisper can transcribe 「未定稿」 as 「みていこう」 and only the timing is borrowed; the spelling stays correct.

Output: analysis.json

The whole structure of one song goes into this one file. An excerpt from real output:

{
  "schema_version": 1,
  "duration_s": 248.36,
  "bpm": 130.81,
  "beats":     [0.0, 0.46, 0.92, ...],   // 537 entries
  "downbeats": [0.0, 1.86, 3.71, ...],   // 135 entries (every 4 beats)
  "sections": [
    { "label": "block1", "start": 0.0,   "end": 22.74, "vocal": true },
    { "label": "block2", "start": 22.74, "end": 36.96, "vocal": true },
    ...
  ],
  "lyrics": [
    {
      "line": "予定はまだ　鉛筆書き",
      "start": 0.0, "end": 3.94,
      "words": [
        { "w": "予定はまだ", "start": 0.0,  "end": 2.26 },
        { "w": "鉛筆書き",   "start": 2.26, "end": 3.94 }
      ],
      "matched_ratio": 1.0,
      "low_confidence": false
    },
    ...
  ]
}

The fields to note are matched_ratio and low_confidence. "How much of this line matched" is kept per line, so only the suspicious lines need a human look. In the song above 73 of 85 lines got timings; the rest were section markers like [Intro].

sections is built from the lyric blocks. Each block of singing becomes blockN (vocal: true), and where there is a sufficient gap between blocks an instrumental section is inserted as intro / interludeN / outro. The sections always cover the timeline without gaps — every section must be assigned cuts, and a hole would break the design.

Downstream reads only this file. Which 60 seconds to take (the window) is decided from sections and downbeats, cut boundaries are aligned to downbeats, and lyrics guarantees that burned-in lyrics coincide with when they are actually sung. Nothing after this point re-analyses the audio.

Design (step 3)

This is the conversation between micha and Claude. Looking at the beats and lyric timings in analysis.json, we decide where to cut, which cuts get lip-sync, and which key visual goes where. If micha has a design document made with GPT (shared header + cut table + lip-sync markings), that is ported first. A design that exists up front finishes better than leaving it to automatic generation.

The decisions are written into a per-song builder, make_<s>.py, which generates shotlist.yaml (the cut list: id / start / end / mode / kv / typography) and prompts.yaml (a prompt per cut). Rounding cut lengths onto 17k+5 is done by framing.py.

Key visuals (step 4)

uv run python -m mvpipe.codexkv ../songs/<s> --spec multicut_kvgen.yaml

The missing ones are generated by Codex CLI. The spec is multicut_kvgen.yaml (cut definitions + style_tail); overriding style_tail also lets you burn lyrics in. Alongside the output, kv_review.html is generated: hover for full size, click for a new tab, and a comment box per cut.

Here micha checks every image. Above all the spelling of burned-in text. Writing a comment on a cut to fix assembles the regeneration command automatically.

uv run python -m mvpipe.codexkv ../songs/<s> --spec <spec> \
  --force --cuts c01 --note c01="..."

Build jobs and preflight (step 5)

uv run python ../songs/<s>/make_<s>.py
uv run python -m mvpipe.jobs build ../songs/<s>/
uv run python -m mvpipe.verify preflight ../songs/<s>/

jobs build assembles job JSON from shotlist and prompts. In the process it picks the acceleration lane per mode, extracts the audio segments for ref2va as 32 kHz wav, pins the seeds, and checks that the references exist.

preflight prints the pre-launch checklist: orientation (portrait/landscape), presence of references, length, and estimated generation time, in one table. Until micha reads this and approves, the GPU is not billed at all.

Submit and generate (steps 6–7)

uv run python -m mvpipe.sync push ../songs/<s>/ --shots s001 s002 ...
uv run python -m mvpipe.runpod launch ../songs/<s>/ [--cloud secure]

push sends references and jobs to jobs/pending/ on Drive via rclone. A guard trips if done is not empty. launch tries community first and falls back to secure if none can be had. The balance is printed at launch.

On the Pod, start.sh installs ffmpeg and rclone, confirms torch 2.11.0+cu128, then starts h3_worker.py. The worker downloads the model from HuggingFace, brings up ComfyUI, takes pending, generates, writes to results/ and moves the job to done. When the queue is empty it deletes its own Pod (AUTO_TERMINATE).

On a crash, jobs in running are returned to pending, so nothing is lost. A failed job leaves failed/<shot>.error.json with the stage, the error and the elapsed seconds.

Never push to a song that has a running Pod. The mirror is one-way, Pod → Drive, so the write-back collides.

Collect and verify (step 8)

uv run python -m mvpipe.sync status ../songs/<s>
uv run python -m mvpipe.sync pull ../songs/<s>/
uv run python -m mvpipe.verify results ../songs/<s>/

verify results checks every clip with ffprobe: whether length, resolution and frame count match the spec, and whether the seed matches meta.json. Reject by machine what can be rejected before anyone looks — that is the order.

Then micha looks at review.html (the clips laid out in a grid). Selecting the cuts to fix assembles the retake command.

uv run python -m mvpipe.retake ../songs/<s> s0XX ...

A retake advances the take counter and re-queues on Drive. No push is needed; if the Pod is running it picks the job up as-is.

Assemble (steps 9–10)

rm -rf ../songs/<s>/out/intermediate
uv run python -m mvpipe.post ../songs/<s>/

Each clip is trimmed to its slot, concatenated, and the Suno master audio (48 kHz) is laid back in. The audio H3 generated is discarded completely here. Matching mouth to voice is the generator's job, but the sound itself is the master's.

If lyrics go on screen they are designed as ASS subtitles: positions rotate, long lines auto-shrink the font, and anchors are clamped at the screen edge. Cuts whose key visual already has the lyric burned in (typography=model) are skipped. Finally a LUT and grain are applied and the file is written.

Deleting out/intermediate/ is mandatory. Intermediate files are cached, and re-running without clearing them mixes in the previous result.

06QC — where a human looks

The purpose of the automation is not to remove the human but to narrow down where the human must look. There are five gates: four are human eyes, one is a machine check of every clip. Only the first two (kv review and preflight) can stop the billing; the rest inform the decision to remake.

Three further operating rules are fixed. Every RunPod run report includes the amount spent and the balance. A key visual with lyrics burned in is used only on the cut where those lyrics are actually sung. And out/intermediate/ is always cleared before re-running post.

07Results

Measurements from three songs in real production. Every number is measured; nothing is estimated.

Not December was made entirely in ref2va and cost $1.00. Restricting ref2va to the singing cuts on the next two brought that down to $0.51 and $0.29. Scarlet has only 10 cuts and finished in 23 minutes for $0.29. Retakes were zero on the latter two.

Key-visual generation costs $0 (within the ChatGPT plan). The ones micha makes with GPT-Image2 are also within the plan, so the variable cost of image generation is zero. The only real variable cost per video is GPU billing.

08Options that were dropped

For anyone using this as a reference, here is what was tried and rejected.

• Multi-cut (several cuts in one generation) — both the 8-second and the 14.4-second windows timed out. Three attempts, $3.18 spent, zero seconds of footage. The implementation remains as multicut.py but is unused
• EasyCache on ref2va — slower than Spectrum, so rejected (73 frames: 185.8 s → 195.5 s)
• Watching Pod telemetry — CPU/GPU utilisation on community Pods can disagree with reality, so it was removed from the health judgement
• H3's generated audio — not used. Sound is unified on the Suno master

09What comes next

H3 is still moving from R&D to established production, so investment in fixed costs waits until the decision on which method to standardise on is made. For key visuals, the ones micha makes with GPT-Image2 are the mainstay for now; how much to hand to automatic generation inside the pipeline is still under consideration.

The immediate issues are making the Drive mirror bidirectional (it is currently one-way, Pod → Drive, so you cannot push to a song while it is running) and getting cutcheck.py, which measures the per-cut timing accuracy of multi-cut, into operation.

How individual songs were made is written up in Suno × MiniMax H3 production notes, Production log 2026-08-12 and 流転 (Ruten) ─ Amor manet production notes (all in Japanese). 流転 (Ruten) in particular is a full day spent testing the cut-to-cut joins mentioned in this article.