The Permission Slip
A while back I asked in this space what would happen if Dario Amodei was wrong. I want to come back to that, because I think the question matters more now than it did then, and for a reason that has nothing to do with whether I like Dario or his company. I do, for the record. That’s not the point.
The point is a document.
In Machines of Loving Grace, Amodei made the case that scaling compute would eventually solve essentially every hard problem in artificial intelligence. Buried in that optimism — or maybe not buried, maybe right out in the open — was a quiet absolution. Hallucinations, the embarrassing tendency of these systems to state falsehoods with total confidence, would take care of themselves. Make the models big enough, train them long enough, and the problem dissolves. You don’t have to solve it. You just have to wait, and spend. And so the entire AI industry breathed a sigh of releif.
I have spent forty years watching this industry, and I know a permission slip when I see one.
Because that is what the essay became, whatever Amodei intended. It gave every other person writing nine- and ten-figure checks a reason not to worry about the one thing that should worry them most. The hallucination problem is the difference between a clever toy and a system a hospital or a bank or a court can actually rely on. It is the whole ballgame for enterprise AI. And the prevailing wisdom, blessed from the top, is that you needn’t address it directly. Scale will provide.
Look at where the money is going and you can see the permission slip being cashed. Stargate, half a trillion dollars. The hyperscalers, tens of billions each per year. The Anthropic–Akamai arrangement, nearly two billion more. The collective bet of the wealthiest companies in the world is that you fix intelligence — including its honesty — by buying more of it. The data center operators are happy. The chip vendors are ecstatic. The labs raising money at valuations with too many zeros are happy. Everyone in that chain has the same incentive, which is to believe that the answer is more.
The customers who will eventually pay for all of it are the ones who should be asking whether any of this is true.
Here is why I think it isn’t. A small company I helped start, 2Brains Inc., set out in 2022 to solve hallucinations — before ChatGPT, before the scaling consensus hardened into received truth, back when the polite assumption was that the problem was simply insurmountable. We did not solve it by waiting for bigger models. We solved it architecturally, by separating the part of the system that generates language from the part that retrieves and verifies facts, and reconciling the two before anything reaches the user. It runs on ordinary processors. It is cheap. And on the industry’s own benchmark for this kind of faithfulness, it more than doubles the published baseline, with no fabricated facts in the verified case at all.
I am not telling you this to sell you anything. I am telling you because of what it implies about the trillion-dollar bet.
If a handful of people in Virginia and Kansas could solve hallucinations with an architecture and a CPU, then one of two things must be true about the scaling story, and neither is comfortable for the people cashing the permission slip.
The first possibility is that scaling will not cure hallucinations at all. That the models get bigger and more fluent and more useful, and continue, reliably, to lie. In that case the largest companies in the world are spending a fortune chasing a cure that is not coming, and the absolution Amodei offered turns out to have been the most expensive sentence in the history of the field.
The second possibility is that scaling will eventually reduce hallucinations — but only by spending enormous sums to arrive, the long way around, at the same place a small company already reached by design. And if the route the giants take passes through the architecture we built and protected, then “scale will solve it” turns out to mean “scale will eventually reinvent something that is already spoken for.” That is not a threat. It is just what the words mean when you follow them to the end.
I find the whole thing clarifying, actually. For three years the conversation about AI has been organized around a single article of faith, which is that the answer to every problem is more compute, and the people who benefit most from that faith are the people best positioned to spread it. It is a remarkably convenient theology. It asks the believers to spend, and it asks the skeptics to wait, and it never quite gets around to the question of whether the central promise is true.
I asked once what happens if Dario is wrong. I am increasingly convinced the more interesting question is what happens when the rest of them realize he might be — and that the bill for finding out is already coming due.
Robert X. Cringely is a co-founder of 2Brains, Inc.
I'm down at South Waterfront this morning, waiting while my better half consults with the medicos, so I've got an hour to kill. I spend my time reading
Idoru by William Gibson, napping and walking. While I'm reading I overhear man on the sidewalk talking to someone. I didn't really listen to what he was saying but you could tell he was excited about whatever it was he was talking about. I think it was because he had someone who wanted to hear what he had to say. People like to talk and it's glorious when somebody listens. This might be AI's biggest market - providing attentive listeners to people who want to talk. In that case perfect honest could be a handicap.
I ordered
Idoru without sampling it based on
Tam's recommendation. Also, I've read other books by Gibson, he's pretty great. I don't know when
Idoru was written but it sounds like he's imagining what our present day will be like, but he's imagining it 20 or 30 years ago. In any case it feels like it's spot on.
.png)
BY TYLER DURDEN FRIDAY, AUG 30, 2024 - 08:30 PM 
