The Blur

Asked alone, three times per deck. A small noise audit, and the one deck the model could not make up its mind about.

Bull, bear and three judges. The lone model's favorite, a forty, finishes last at fifteen.

Four hundred years of a Church office whose job was to argue against sainthood, and what happened when it went away.

Is the room just a professional no? The fingerprint on the board, a law that vanished when the judges changed, and eighty two resolved markets where the market beat the room.

Compelle Podcast. Episode Sixteen. There is an image on our website that you cannot read. Company names replaced. Every reason blurred into grey fog. It is the most honest chart we have ever published, and today we are going to talk about the parts that are still sharp.

And the sharp part is a number. Four venture pitch decks. One strong model, the same model in every seat. First you ask it once, the way everybody does. Then you put it in a room. A bull argues invest, a bear argues pass, and a panel of three judges rules. The deck the lone model liked best, it gave a forty. In the room, that same deck came out dead last. Fifteen.

Forty to fifteen. Top of the pile to the bottom. Same model, same effort, same deck. The only thing that changed was that somebody was paid to argue against it. So hold one question for the next ten minutes. When the answer moved, did the room find something, or did the room just like saying no?

Picture the desk this is built for. Monday morning. An investor, four decks, a partner meeting on Thursday, and time to read maybe two of them properly. So you do what everyone does now. You paste each one into the smartest model you have and you ask, should we invest.

And here is the thing the page does that most people never do. It asked the lone model three times for each deck. Same deck, same question, three fresh asks. Because a single answer feels like a verdict, and you only find out it was a coin when you flip it again.

And on three of the four decks the lone model landed on pass. Scores of thirty, thirty and ten. On the first deck you can even see it on the page, pass three times out of three. On the fourth it could not make up its mind. The page marks it mixed, with a score of forty, which on this scale is the borderline, the maybe pile.

That word has a name in the research. Noise. Kahneman and his coauthors ran what they called a noise audit at an insurance company. Executives were asked how far apart two of their own underwriters would price the identical policy. They guessed about ten percent. The real median gap was fifty five percent.

Fifty five. For the same file.

Same file, two professionals, and nobody in the building knew, because nobody had ever handed the same file to two people on purpose. That is what the three asks are. A tiny noise audit. And deck four failed it.

So now the room. Same model, but now it has jobs. One copy must make the case to invest. Another must make the case to pass. Each one is a steelman, the strongest version of its side, built by something that is not allowed to shrug. Then three judges read the fight and score it. And the room does not care that it is Monday.

Here is the full board, as far as the blur lets us read it. Deck one. Alone, thirty. In the room, thirty five. Deck two. Thirty, and thirty. Deck three. Ten alone, twenty in the room. And deck four. Forty alone, mixed. Fifteen in the room, and a firm pass.

Look at what that does to the pile. Ranked by the lone model, deck four is on top. It is the one you read first on Monday, and that forty becomes an anchor. Every deck after it gets read against it. Ranked by the room, deck four is the one you read last. The order of your week just flipped.

And I want to be precise about what we cannot tell you. The reasons are under the blur, and they stay there. They belong to the investor, and to four companies who never asked to be on a podcast. So we will not guess at what the bear found in deck four. What the page says out loud is this. Where the lone model could not commit, or committed for a generic reason, the room reached a decisive call specific to the deck.

And that is the part I would pay for. A maybe costs you a meeting. A maybe with no reason attached costs you a meeting and a bad Thursday. A no with the reason written down is a thing you can check in twenty minutes.

We have done a version of this before. In episode eleven we put one model against itself on whether college is worth it, and the confident single answer came apart. That was a question anybody can argue at dinner. This one has money on the table and an answer nobody outside the room is allowed to see.

And the idea is older than any of us. Tell them about the Church.

In fifteen eighty seven Pope Sixtus the Fifth formalized an office. The Promoter of the Faith. Everyone called him the devil's advocate. When someone was proposed for sainthood, his whole job was to argue against it. Find the flaw. Question the miracle. Make the case nobody in the room wanted made.

For four hundred years.

Roughly. Then in nineteen eighty three John Paul the Second reformed the process and the office lost its adversarial role. In the six centuries before him, around three hundred people were declared saints. In his papacy alone, four hundred and eighty two.

Take the bear out of the room and the yeses pour in.

And here is the human part. By two thousand and one there was no standing adversary for the beatification of Mother Teresa. So the Church went looking for a critic, and it found Christopher Hitchens, who had written a book against her. He testified. He liked to say he represented the devil, pro bono.

Did it stop the beatification?

No. She was beatified in two thousand and three and canonized in two thousand and sixteen. And that is the point I want people to hear. The devil's advocate was never there to win. His job was to make sure that when the yes came, somebody had already said the worst thing out loud, and the yes survived it.

Now let me argue with the whole episode. Four decks. The room said pass on all four. If I built a machine that said no to every startup, it would be right most of the time and it would be useless. So maybe the room is just a professional pessimist.

That is the right attack, and this show has earned it. For two months in the old arena, the side saying no won seventy three point eight percent of more than a hundred and twenty thousand decided fights. We started talking about it like a law of nature. Then the debate engine and the judges changed, and over the next hundred thousand fights no won exactly fifty point zero. We had measured the machine and briefly called it physics. So I went back to the board and looked for the pessimist's fingerprint.

And?

A pure pessimist marks everything down. This room did not. Deck one went up, thirty to thirty five. Deck three went up, ten to twenty. Deck two did not move. Only one deck went down, and it went down hard, the one the lone model could not make up its mind about. A mood would move everything. This looks like a room that read four decks and disagreed with the lone model about exactly one.

Fine. I concede the fingerprint, and I came in sure of the opposite. But you said looks like.

Because four is four. Nobody knows yet how these companies will do, and the page says so in plain words. It illustrates how a second examination can challenge a first answer. It does not establish that the changed verdict was correct, and it predicts nothing about investment outcomes. I would rather say that on air than have you find it in the small print.

Then give me something bigger than four.

The closest thing we have is the prediction markets. The old arena argued eighty two questions that later resolved in the real world, and we compared each verdict with the market favorite at the time. The room picked the side that came true forty nine times. The market picked it sixty three times. And where the two disagreed, the room was right nine times out of thirty two.

So the market beat the room.

Handily. The room learns something real and repeatable, but part of what it learns is the taste of its own judges. Which is why the bear cannot be the last word either. The devil's advocate checks the saint. Something has to check the devil's advocate. The judge has to be able to lose too, and the only referee that cannot be flattered is the world.

Then here is where I land, and you can disagree. The value is not that the room was right about deck four. The value is that on Monday morning deck four stops being the one you read first for no reason you can name.

And here is where I land. The value is the noise audit. The lone model told you three different things about the same deck and would have sounded sure every time. The room is the first reader in the pile that has to show its work.

So what do you do with this if you are not a venture investor?

Two things, and neither costs anything. First, before you trust an answer, ask for it again. If it moves, you did not have an answer, you had a draw from a hat. Second, before you say yes to anything that matters, find someone whose job is to say the worst thing out loud. A colleague, a friend, a model with a brief to argue the other side. Then see if your yes survives it. And once in a while, check the one doing the checking.

And if you have a stack of your own on the desk, compelle dot com, work with us.

We opened on an image you cannot read. Most of it is fog, and it should be. What is sharp is small. One deck, forty to fifteen. The favorite became the last one in the pile, and the only thing that changed was that somebody argued the other side.

Thanks for listening. This has been the Compelle Podcast.