The Rotoscope and the Art of Tracing a Real Performance

On December 6, 1915, Max Fleischer filed a patent for a new way to make cartoons: film a real person moving, project the film one frame at a time, and trace each pose by hand. Any brand weighing AI video built from real footage is buying a descendant of that idea. Video-to-video generation restyles a filmed clip. Motion reference lets a recorded performance drive a generated character. Both are rotoscoping with a model in place of the pencil, and the rotoscope’s century-old arguments over quality, consistency and credit apply to them almost line for line.

A clown, a projector and a pane of glass

Fleischer’s patent, granted on October 9, 1917, describes the method plainly: photograph a moving figure, project the photographs one after another onto translucent sheets, and draw the cartoon on the other side. His studio’s own account adds the glass: film projected onto the back of a small panel, paper on top.

The performer was family. Max’s brother Dave had worked as a clown and served as the live model for the character later named Koko, star of the Out of the Inkwell films. Film then ran at 16 frames per second, so every second on screen meant 16 separate drawings. The result was a cartoon that moved with the weight and timing of a real body, and all of it came from Dave.

Motion reference in AI video works on the same principle. The model takes the timing, weight and path of a filmed body and builds a generated character around it. It can change the costume, the face and the world. It cannot supply a sense of weight the performer never gave it. Fleischer’s setup also shows how to shoot for it: one performer, a fixed camera, a readable silhouette. A body half hidden behind a counter, or a hand crossing the face, leaves the model less to follow and more to guess.

Disney used the footage to study

For Snow White and the Seven Dwarfs, which premiered in 1937, Disney filmed a young dancer, Marjorie Belcher, later known as Marge Champion, acting out Snow White’s scenes and dances in costume so the animators could caricature her movement. The Walt Disney Family Museum notes that similar footage of the dwarfs was filmed “simply to suggest possible attitudes and actions for the animators’ benefit.” The animators exaggerated and redrew.

That gap between tracing and studying is the main dial in video-to-video work. Many workflows let the director set how closely the output follows the source clip. Held tight, the generation keeps every gesture, flaws included. Held loose, the model gets room to exaggerate like a Disney animator, and room to drift from what was filmed, including the shape of the product. Neither setting is right in general. The choice belongs in the shot list, before anything is generated.

Champion was still a teenager when she started. Watching the finished film years later, she said she realized “every single movement was mine.” The same holds for anyone whose motion drives a generated character, whether a dancer, an employee or an actor. They are performing even when the face on screen belongs to someone else, so their consent and pay should be settled before the reference shoot. Synthetic presenters raise the same question.

Tracing goes big

Tracing kept returning whenever drawing alone could not fake the effect. For Star Wars in 1977, the glowing lightsabers began as mattes of the prop blades used on set, rephotographed and colored frame by frame, according to Industrial Light & Magic. For Return of the Jedi, Peter Kuran’s company rotoscoped every frame of the duel between Luke and Vader: 102 lightsaber shots in all.

Ralph Bakshi made The Lord of the Rings (1978) by shooting the story in live action first, then animating over it. He wanted illustration rather than cartoons, and he voiced the objection critics had always raised: “if you’re not going to be cartoony, why animate?” The question fits any restyled clip. If the footage already works, the style has to earn its place.

Rotoscoping also survived as a visual effects chore: cutting a moving subject out of its background so it can sit on another one. AI has already taken over part of that chore. On Everything Everywhere All at Once, visual effects artist Evan Halleck worked with AI-assisted tools, including on the film’s rock-universe scene. Cutting out characters and placing them on a plate, he told Variety, took “minutes versus what takes half a day.” For a brand, the same step decides whether a filmed product sits convincingly in a generated setting. Edges are where it breaks: hair, motion blur, glass, reflections. Review the composite at full size, on the hardest frames.

Painting over real life

Richard Linklater carried the technique into the digital era. For Waking Life (2001), he filmed real people talking, then artists drew over the footage on computers with interpolated rotoscoping software developed by Bob Sabiston. Roger Ebert wrote that the approach “captures little moments of real life.”

A Scanner Darkly (2006) used the same method with a cast that included Keanu Reeves. The actors performed in plain grey sweat suits, and the film’s shape-shifting scramble suit was drawn on afterward. Linklater estimated more than 500 hours of work for each minute of animation. Reeves said that without his face to carry information, he moved a little bigger. A reference performance for a model calls for the same instinct: play to what will survive the restyle.

Drawn by hand or interpolated by software, a traced line has to hold steady from one frame to the next or the image crawls. Generated video fights the same battle. A restyled clip can shimmer at the edges, a logo can change a letter mid-shot, a face can drift between two shots generated separately. In A Scanner Darkly, constant change was the point of the scramble suit. Everywhere else it is a defect, so a director keeps clips short, locks the look with the same reference images across the sequence and checks the result frame by frame.

110 years of tracing reality

Key moments of the rotoscope, from patent to generated video

Timeline of the rotoscope

Year	Milestone
1915	Rotoscope patent filed: Max Fleischer, December 6
1917	Patent granted: October 9
1937	Snow White: Live-action reference for animators
1977	Star Wars: Saber glow colored frame by frame
1978	The Lord of the Rings: Bakshi shoots it live, then animates
1983	Return of the Jedi: 102 rotoscoped lightsaber shots
2001	Waking Life: Digital rotoscoping over real footage
2006	A Scanner Darkly: 500+ hours per minute of animation
2007	Academy rule change: Both films ruled out of the category
2025	Academy rule on generative AI: Human authorship is what counts
Now	Generated video: A real performance drives the image

Is tracing real animation?

In June 2007, the Academy tightened its rules for Best Animated Feature around frame-by-frame character animation, which ruled out films like Waking Life and A Scanner Darkly. According to Animation World Network, the branch felt the digital rotoscoping was “not crucial enough in shaping the animated performances.” In 2010, the rules added a line of their own: “Motion capture by itself is not an animation technique.” In plain terms, the Academy decided those performances belonged to the actors.

In April 2025, the Academy wrote its first rule on generative AI. The tools neither help nor harm a film’s chances of a nomination, it said, and it will weigh “the degree to which a human was at the heart of the creative authorship.” That is the rotoscope question asked of a new pencil. In AI-assisted animation, authorship lives in the choices: who performed, who designed the look, who picked, corrected and cut each generation. A brand needs the same accounting, shot by shot, of what was filmed, drawn and generated, for credits, for disclosure and for rights.

What to ask before the reference shoot

If real footage will drive the generated image, treat the reference shoot as the real shoot: cast it, light it and direct it with care. Ask to see the reference next to the result, so you can judge what the model added and what it simply copied. Ask how closely each shot follows its source, and check products, logos and faces frame by frame for drift. Get written consent from everyone whose movement, face or voice is used, for this use and for a set period, and decide up front how they are credited and paid. Fleischer’s device made the drawings. Dave’s clown act made them worth watching.