How to build a keyframe-to-video pipeline that keeps your shots consistent

Generating video clip by clip almost always drifts. You prompt the same character into five consecutive shots and by shot three she is wearing a slightly different coat, the kitchen behind her has gained a window, and the shadows have swapped sides. Each clip looks fine in isolation. Cut together, the scene falls apart. This is not a bug in any single model. It is what happens when every generation starts from scratch with only text as its anchor.

A keyframe-to-video pipeline replaces that text-only anchor with an image. You generate a still first, approve it, and feed it as the starting frame for the video generation. The still holds your character design, your set, your light, your lens character. The motion model then has something concrete to stay faithful to. This workflow costs a little more time at the planning stage and saves a disproportionate amount of time in the back half, when you would otherwise be regenerating or cutting around drift in the edit.

The basic shape of the pipeline

The pipeline has three stages: image generation, review and lock, video generation from image. Nothing moves to stage three until stage two is complete. That sounds obvious, but the temptation to skip straight to motion is where most projects go wrong.

Stage one: generate your keyframes as stills.
Write a prompt for each distinct shot setup in your scene. Think in terms of camera position and subject state, not mood. A keyframe prompt should answer: where is the camera, what is the subject doing, what is the light source and direction, what is in the background. One setup, one still. Generate several variants per setup and pick the best one before you touch video generation at all.

Stage two: review for consistency across setups.
Lay your approved stills side by side before generating any video. Check that the character reads as the same person across all angles. Check that background architecture, props, and light direction are coherent. Fix at the still stage. Regenerating a still costs a fraction of what regenerating a video clip costs.

Stage three: use the approved still as your start frame.
Feed the still into your image-to-video model as the first frame input. Your motion prompt now only needs to describe what changes: the action, the camera move, the duration. It does not need to re-describe the character or the set because the image is already doing that work.

Writing the keyframe prompt

A keyframe prompt is a cinematographer's setup note, not a mood board caption. Structure it in layers:

```
[Subject description]: [action or pose]
[Camera position and lens character]: [angle, height, rough focal length feel]
[Light]: [source, direction, quality]
[Environment]: [specific background elements, depth]
[Technical]: [aspect ratio, photographic or rendered feel, grain if relevant]
```

A concrete example:

```
Woman in her late forties, short grey hair, dark green canvas jacket,
standing at a cluttered kitchen counter holding a mug with both hands.
Medium close-up, slight low angle, 50mm equivalent.
Overcast window light from camera left, soft shadows.
Kitchen background: open shelving with mismatched crockery, a yellow wall,
no visible windows on the right side.
Photographic, slight grain, 16:9.
```

Notice that the yellow wall and the closed right side of the frame are specified. Those are the details that drift between shots if you leave them to chance. Lock them in the still, and the video model inherits them.

Writing the motion prompt once you have the start frame

With the still loaded as your first frame, your motion prompt can be short and precise:

```
She lifts the mug slowly and takes a sip. Camera holds steady.
Subtle ambient movement: steam rising from the mug, slight breath movement.
No camera push. Duration approximately 4 seconds.
```

You are not re-describing the room. You are not re-describing her jacket. The image carries that. The motion prompt is purely about change over time.

The before-and-after difference you will see

With a text-only workflow, you will typically need three to five regeneration rounds per clip to get a character who matches her previous shot. With a keyframe pipeline, the first or second generation of each clip is usually usable because the visual anchor is already there. The gains compound across a scene: a five-shot scene that previously took twenty-five to thirty clip generations to resolve might settle in eight to twelve. The edit also becomes faster because you are choosing between good options, not hunting for one that is merely acceptable.

The discipline the pipeline enforces is separation of concerns. Appearance decisions happen at the still stage. Motion decisions happen at the video stage. Mixing them, which is what pure text-to-video prompting asks you to do, is why consistency is so hard to maintain across a scene.

What to try in your next session

Pick one scene you have been planning and write keyframe prompts for each of its shot setups before you open your video model. Generate and approve all the stills first. Then generate the video. Compare how many regeneration rounds each clip takes against your usual text-only approach, and use that number to calibrate how you budget credits on the next project.