Skip to content

Blog

The model made the shot. It didn't make the scene.

A model can generate a shot with sound. A scene needs one acoustic world. Lunar Canoe makes the soundtrack from the edit.

A weathered silver spacecraft lies open among trees beside a lake, with morning light filtering through the forest.

Cut from a fisherman’s hands to a gull on a bollard to the wide harbour. The camera has changed three times. The harbour has not.

That distinction is easy to miss when making AI video, because the model makes each shot as a small, complete world. It gives you picture, action, and often sound in one go. The result can be startling in isolation: waves where there should be waves, boots where there should be boots, a passing engine in the distance.

Then you put the shots next to each other. The camera cuts away, and the world starts over.

The water changes character. The gulls move to a different part of the sky. The room tone rises or disappears. Nothing is necessarily wrong with either clip; they were simply never made to share an acoustic space.

A shot is not an acoustic unit

Picture and sound have different jobs in an edit. A cut can move the viewer from a close-up to a wide shot, from a face to the thing being watched, or from one side of a room to the other. It is supposed to change what we see.

Sound often does the opposite. It tells us that all those images still occupy the same room, street, train carriage or shoreline. It can arrive before the picture does and remain after the picture has gone. The camera has permission to cut. The world usually does not.

That is why native audio from a video model is useful raw material but a poor default soundtrack. It is generated on the model’s natural unit: the clip. A film is made of a different unit: the scene.

The usual response is to mute everything and find sound later. That is sometimes right, but it turns a basic act of scene-making into another handoff: export the cut, find or record a bed, time it, mix it, and hope it still fits when the picture changes. A better workflow should understand the cut without asking the editor to rebuild the world around it by hand.

Generate the picture first. Generate the world second.

Lunar Canoe builds picture as short, independently replaceable shots. That is how it keeps one bad take from disturbing the rest of the film. Once a contiguous run is working, it can make a silent render of that run and generate one soundtrack from the assembled picture and a description of the sound.

The input is not “a gull on a bollard” or “hands tying a rope.” It is the actual sequence: hands, gull, harbour. The generated result is one audio asset that spans the run, rather than several miniature soundtracks that happen to touch at their edges.

That distinction changes the prompt too. Instead of asking every shot to re-invent its own atmosphere, you can describe the place: waves against the harbour wall, gulls overhead, a distant engine, no music. The model gets the changes of picture as context, while the sound gets to remain in the same world.

The soundtrack belongs to the edit, not to a shot

The result lands in the project library and can be placed on the music lane, anchored to the first shot of the run. From there it behaves like any other piece of an edit: preview it, set its level, and decide whether the native audio in those shots should remain.

That last choice matters. A generated soundtrack is good at atmosphere and events. It is not a reason to throw away a useful line reading or a dialogue shot. The point is not to replace every sound a video model made. It is to let the editor decide which sounds belong to the shot and which belong to the scene.

It also means the soundtrack is honest about its dependency on the picture. If the run changes after sound is generated, Lunar Canoe marks the soundtrack as stale. A new cut may deserve new timing, a new door slam, or a different swell of water. Sound should follow the edit; it should not quietly pretend it still does.

The model made dailies. You are making a scene.

AI video gets much better when we stop asking a single generation to be the finished film. A clip can be a replaceable take. A pinned frame can be a contract at a cut. A soundtrack can be a response to the sequence that those cuts create.

The model made the shot. It did not make the scene. That remains the editor’s job — and it is why the sound has to be generated from the edit, not separately inside every clip.

Written by the people building Lunar Canoe, a storyboard for AI video. More posts on the blog index, or subscribe by RSS.