AI Video Has a Sound Problem

Turn the sound off during a horror film and the fear drains out of it. Turn it off during a fight scene and the punches stop landing. Sound is the part of a film nobody notices, until it is missing. It is also the part most AI video skips.

A lightsaber made from a projector and a broken TV

Nothing in Star Wars (1977) sounds like the real world, because none of it existed. Sound designer Ben Burtt built the lightsaber hum from an idling film projector combined with feedback from a broken television set. The blaster started as the sound of a hammer hitting the guy-wire of a radio tower. Burtt received a special Oscar for the film’s alien, creature and robot voices.

A glowing stick feels like a weapon because of what you hear. Take the hum away and it is a prop.

The shark you barely see

On Jaws (1975), the mechanical shark kept breaking down, so Steven Spielberg showed it far less than planned: it does not fully appear until the 81st minute of a 124-minute film. John Williams’s two-note theme does the work instead. Spielberg reportedly thought it was a joke when he first heard it. Williams won the Oscar for the score.

For most of the film, the audience is afraid of a sound.

How much does sound weigh?

George Lucas is widely quoted as saying that sound is 50 percent of the moviegoing experience. Research points the same way, with nuance. In 1991, W. Russell Neuman, Ann Crigler and V. Michael Bove tested 367 viewers and found that programs with better audio were also rated as having better picture quality, although in only one of the three programs tested. A modest result, and a telling one: what people hear changes what they think they see.

What a real soundtrack is made of

A finished film does not have “a sound”. It has layers, mixed so that nobody notices them: voices, sound effects for every action on screen, an ambience that tells you where you are, music, and silence used on purpose.

Five layers under every scene

Schematic timeline of a mixed soundtrack

Why AI video sounds cheap

Most generated video started out silent, and a lot of it still reaches the screen with one stock music track laid over everything. Some models now generate sound along with the picture. That helps, but it brings the same weakness as generating images shot by shot: nothing ties one clip to the next. The room tone changes at every cut, the footsteps do not match the floor, the ambience of an office sounds like an airport.

Viewers rarely say “the sound is wrong”. They say the video feels cheap. And they are right: the image has been directed, the sound has not. The visual tells are covered here.

What it means for a brand

Treat sound as half the film, not a finishing touch added the night before delivery. Voice, effects, ambience and music should be designed together, with the same care as the picture.

Two quick tests before you approve a video. Close your eyes and listen: can you tell where you are, what is happening and how you should feel? Then watch it muted: does the picture still carry the story? A good film passes both. A great one needs both at once.