Case Study Zero
Most explainer videos are a talking head that barely changes. The host delivers the same scripted line four or five times, the takes come out nearly identical, and one of them has to be chosen. That is editorial decision making at its most particular, and it makes one of the most unusual tests of taste and judgment we could find.
Abstract. We used footage from an art history YouTube explainer series, recovering 2,974 talking head editorial choices from five episodes. We did it by aligning transcripts of the raw footage against the finished cuts, with no project files and no manual labelling. In the 183 cases where a line was shot more than once and exactly one take survived, nothing inside a take predicted the choice: not the words, not pitch, energy or pace, not eyeline or expression. The taste sits only in how a take stands against its rivals. When we wrote this into a file and handed it to a general-purpose model, it moved that model from chance to the host’s own reliability.
A model with a taste emulator beat a blind model at editorial decisions by 16 points.
Identical model, identical 183 choices, identical prompt. The only difference was whether the host’s taste came with it. Blind it scored 37.7%, barely above the 32.0% you get by guessing. With the file, 54.1%.
What was built
Every take that did not make the cut was rejected by one person for reasons that were never recorded. No project files, no edit decision lists, no notes. The only surviving trace of the decision is the difference between what was shot and what went out.
So we transcribed every raw camera take and every finished cut, then aligned the two word streams. Anything that survived into the final is kept, everything else is cut. The check that the labels are sound is how much of each finished episode traces back to an identified moment in the raw.
| Episode | Raw segments | Kept | Final traced to raw |
|---|---|---|---|
| Episode 1 | 329 | 26% | 91% |
| Episode 2 | 562 | 20% | 96% |
| Episode 4 | 430 | 30% | 88% |
| Episode 5 | 537 | 21% | 87% |
| Episode 7 | 1,116 | 26% | 92% |
It holds on all five independently, not only on the episode the method was built against. No human labelled anything.
1. Where the taste actually sits
Because the piece is scripted, what belongs in the episode is settled before the edit begins. The 75% cut rate is arithmetic. A hundred minutes of coverage has to fit a twenty-minute script, so most of it goes regardless of quality.
What is left, once the script holds the content constant, is the one decision he alone makes. The script demands this line, he recorded it four times, and he keeps one. That choice is the taste, and it is the only place to look for it.
Content, speaker, script, camera, lighting and day are all held constant by the way he works. The only thing that varies between the attempts is the performance and the choice. Every earlier study had to construct its contested set. Here he built it himself, by shooting the line again.
2. Nothing inside a take predicts the choice
Three modalities, each scored blind, each holding out whole takes or whole episodes.
| Signal | Blind AUC | Verdict |
|---|---|---|
| The words of the line | 0.508 | null |
| Delivery: pitch, energy, rate, contour | 0.526 | null |
| Vision: eyeline, expression, visible flub | no variance | null |
| Relational: position among its rivals | 0.613 | clears chance |
| Relational and delivery together | 0.624 | clears chance |
Pitch height, range, contour, final fall, loudness, dynamics and speaking rate were measured with voiced-filtered tracking and normalised inside each take. Across 138 contested groups not one separated on both a sign test and a Wilcoxon test. A vision model returned looking into the lens on 97% of 560 frames and never once flagged a flub, which says more about a locked-off camera than about the model. Fifteen text markers mined from the transcripts all landed between 49% and 54%.
Every intuitive explanation fails. He keeps the better-sounding take, the more expressive read, the one where he looked right: none of it survives measurement. The things a person would name as their reasons turn out to carry no weight, which is what every earlier study in this program found in its own decider.
3. One axis carries everything
His taste operates on the comparison between takes, not on any take by itself. Turned into a decision rule and scored the way an agent would use it:
| Rule | Picks his take | vs chance |
|---|---|---|
| First attempt | 21.9% | below |
| Longest take | 30.1% | below |
| Cleanest take, fewest stalls | 35.1% | below |
| Closest to the written script | 40.4% | +8.4 |
| A 17-feature learned ranker | 42.0% | +10.0 |
| Last attempt | 48.1% | +16.1 |
The 17-feature model scores worse than the one-line rule. 183 decisions cannot support 17 parameters. Every minimal model that beats the rule contains a position term, and every model without one falls to chance or below. Adding a second or third independent signal buys about one point, with an interval that swallows it.
Across every other subject in this program a decision broke into several weighted reasons. This taste does not. It runs on a single axis, and adding anything to it earns nothing.
4. The decisive test
One local model, qwen3:14b, run twice over the identical 183 choices at temperature zero. Each prompt lists the competing takes in the order they were recorded and asks which one he kept. The first arm gets only that. The second gets the identical prompt with a 1,455-byte file prepended. Nothing else differs.
A general model handed the raw material and no knowledge of the decider performs at 37.7%, barely above chance. His judgment is not inferable from the content. Handed a 1.5 KB text file it reaches 54.1%, a lift of +16.4 points with a 95% confidence interval of [+6.0, +26.8] that excludes zero.
5. The lift is judgment, not leaked statistics
The full file states its own accuracy and the chance baseline. Supplying those risks telling the model roughly how often the last take wins, which is information about the answer rather than about the decider. So the file was rebuilt with every number stripped out, leaving only the decision unit, the named checks, the rule, the exception and the list of what was tested and found not to matter.
| File given to the agent | Accuracy | 95% CI |
|---|---|---|
| None | 37.7% | [31, 45] |
| Full, including its own accuracy figures | 53.0% | [45, 60] |
| Stripped, judgment only | 54.1% | [47, 61] |
Removing every number changes nothing, a difference of 1.1 points with a 95% CI of [−4.4, +2.2]. The numbers contribute nothing and the judgment carries all of it.
A thinner file, with the named checks stripped out and only the conclusion left in, dropped to 39.9%. Same holdout, same rule, same model. Handing an agent the answer on its own is much weaker than handing it the answer surrounded by the considerations, their limits, and the list of what was ruled out.
6. Meeting the ceiling
The subject estimates his own self-consistency at roughly 50%. Shown the same competing takes twice, he would choose the same one about half the time. That is his estimate and it has not been measured, which is the largest open item in this study.
If it is close, 54.1% against a 50% ceiling means the file has captured most of what can be captured. No model exceeds a decider’s own reliability, so a 90% target was never available. He cannot hit 90% against himself.
When roughly half the rejected takes were acceptable anyway, asking whether the agent picked his exact take sets the bar in the wrong place. The question that matters is whether it picked a take he would accept. That number is necessarily higher, and an assistant who selects an acceptable take has done the job.
What this proved
- One person’s taste can be recovered from work they have already done, with no manual labelling, and the recovery can be verified.
- It is absent from everything inside the thing being chosen, and present only in how that thing stands against its rivals.
- It compresses to a file with no model weights and no training data, and that file is portable to an unrelated general-purpose model.
- The lift survives a proper holdout, survives stripping every number out of the file, and lands at approximately his own reliability.
- The structure of the file matters as much as its conclusion. Ten points sit in the named checks and the list of what does not matter.
What this did not prove
- The ceiling is an estimate, not a measurement. The saturation reading rests entirely on it.
- The agent does not demonstrably beat the rule it encodes. The gap touches zero, and the mechanism proposed for it was tested and rejected.
- One person, one domain, 183 decisions. Two orders of magnitude smaller than Michelin or ICLR, with wide intervals and real variance between episodes.
- Take selection is not editing. This file chooses among attempts at a scripted line. It does not decide what belongs in the piece, and it says nothing about assembly or structure.
Verdict
One person’s taste, recovered from work he had already done, reduced to a single measured axis, written into a file smaller than this page, and handed to a model that had never seen him. The model went from guessing to choosing the way he chooses.
The file holds no weights and no training data. It is a document, readable by the person it describes, and honest about what it cannot do.