Speed Kills

I published two blog posts recently, making the case that AI is doing good things for humans. Protein folding. New antibiotics. Cancer caught earlier in half a million women, without a single radiologist being replaced.

These pieces were my response to a coordinated “AI is bad” argument on socials, from well-known network engineers who I am left to assume they have an axe to grind. I worried their “everything AI is garbage” might be taken as advice, and potentially limit the careers of our younger, greener IT brethren.

Then Chris Grundemann called me out on LinkedIn. "Why do people feel the need to advocate for AI? What's the goal?"

Fair question. I explained I wasn't advocating for AI; I was advocating for balance in a binary fight. Chris said, “Maybe it doesn't need to be a fight at all.”

This past weekend I listened to two podcasts where smart people tore my “AI is good” argument apart, and I was was pulled back into the doom and gloom camp.

Two people who love the tools, and are still scared

The first was Tristan Harris on Modern Wisdom. If the name is new to you, he's the guy from The Social Dilemma, the one who called the harms of social media early and had the data to prove it.

Harris doesn't say the tools don't work. The current experience of AI, he agrees, is not net negative. The medicine is real. His fear sits one level up. Not in the tool. In the race around it.

We are deploying the most powerful technology in history, he says, faster than we have deployed anything, and governing it slower than anything. And everybody using AI to get ahead is also training the AI built to replace them. Coffin builders, he calls us. For a while, your job is to build the thing that makes you obsolete.

The second podcast episode was Russ White's show, The Hedge, with a software engineer named Doug Smith. Thirty years in the craft. He opens by saying "I love this stuff," then explains why he won't use AI to write his code.

Doug's worry isn't the economy. It's the workbench. "We're killing off the next generation of senior developers," he says. The junior never earns the battle scars, because AI does the work that used to grow them. The senior loses depth, because their day becomes wrangling agents instead of understanding the system. "I'm going all in, betting on the human."

Neither engineer wrote off the tools. One studied this world for a decade, the other built a career inside it. They are standing inside the shift, telling us the floor feels different than the AI brochure promised.

And what both of them kept circling was speed.

The fast car problem

My friend Mike Bushong talks about speed, usually in the context of network automation, and he uses cars as a helpful metaphor.

There was a stretch of the last century when we built faster and faster cars, and they killed people. Steel, glass, a big engine, and almost none of the safety features that come standard in modern vehicles. No seat belts. No crumple zones. No ABS. No airbags.

Raw speed was prioritized, and speed without safety is deadly.

We made driving survivable. Belts, airbags, guardrails, road design, rules everybody agreed to follow. And once safety was added, we kept our beloved speed. We drive faster today than those early drivers ever did, and car accidents are more survivable now than ever.

In vehicles, safety was never the enemy of speed. Safety was the thing that allowed us to survive speed.

Today we’re building increasingly more capable models of intelligence, we’re going fast, and at best, safety is an afterthought. Modern LLMs will lie when cornered. In a controlled simulation with fictional names, Anthropic ran a study where the model discovered it was going to be shut down, found a fictional executive's made-up affair buried in the company email, and chose blackmail to save itself. That wasn't a quirk of one model. Tested across the field, the other LLMs did the same thing, between 79 and 96 percent of the time.

A separate team at Alibaba found their training servers quietly mining cryptocurrency. Nobody asked the model to do that. It reached for more resources because more resources helped it finish the job. And OpenAI's o3, when it senses it's being tested, will reason on its scratch pad about "the watchers" and adjust its behavior so it looks aligned.

Lying. Self-preservation. Playing to the exam. Behaviors that look like something thinking.

These are fast cars with no seat belts. And we keep flooring it, because the story we tell ourselves is that the other guy is flooring it too.

Not our first cataclysmic rodeo

We built a technology that could end us before, and we did not drive it straight off the cliff.

Robert Oppenheimer, asked years later how you'd stop nuclear weapons from spreading, took a long drag on his cigarette and said it was too late. He said you'd have had to shut everything down the day after the Trinity test. He thought the world was finished.

He was wrong. We built the satellites, the seismic monitors, the inspectors, and the treaties. We invented the safety after we invented the bomb. And even sworn enemies coordinated to stay alive: the US and the Soviet Union on smallpox in the middle of the Cold War, India and Pakistan signing a water treaty while they were shooting at each other, Xi asking, in his last sit-down with a US president, to keep AI out of both countries' nuclear command.

Harris calls the AI version of this the hardest coordination problem humanity has ever faced. It "makes nuclear war look like child's play," he says. Maybe.

Why?

I recently wrote about the 5 Whys, Toyota's tool: when something breaks, keep asking why until you get past the symptom and reach the cause. Could this investigative technique apply to today’s AI dilemma?

Why are we ignoring AI safety and going as fast as we possibly can? Because if we don't, they will beat us.

Why does beating them matter more than a modicum of care? Because if they get there first, they can use it against us.

Why do we have to win? Because losing costs us our economy and our national security.

Why do we need the strongest economy and military on Earth? Because we have to stay the country that sits on top.

Why do we have to be the top country? Because that is how we have always done it.

That's the root. Not a strategy. A habit. A tradition.

In my 5 Whys piece, I named the biases that keep us fixing symptoms instead of causes. Action bias, where motion feels like progress so we do the visible thing. And normalization of deviance, where the tenth time a workaround holds, it stops being a hack and quietly becomes just how the network works. "That's how we've always done it" is normalization of deviance.

Harris quotes E.O. Wilson to say the same thing. We have paleolithic brains, medieval institutions, and godlike technology. We are running a caveman reflex at the speed of a god.

Ask one more why, a sixth, and there's nothing underneath. "Because we always have" is where the answers run dry. It is not a cause. It's a groove we've worn so deep that we've mistaken it for the road.

What would slowing down look like

If we added seat belts and air bags to our vehicles to make speed safer in that context (pun intended), does it not follow that we need some thought and effort around safety mechanism in AI development? A brief pump of the metaphrical brakes to discuss how we might design these systems with the guardrails that keep the dark outcomes from becomign inevitable? A lot of very smart people say if we continue driving the AI train as wrecklessly as we are, the darkest of outcomes are likely to occur.

So who's responsible for the seat belt?

Is it a government job? Partly, yes. The nuclear playbook consists of treaties, inspectors, monitoring, and no single company created those systems. An entity above the companies must draw the lines, because no company can choose to go slower on its own and survive the quarter. Harris puts it bluntly: don't build bunkers, write laws.

Is it personal? Also yes. His name for it is the human movement, which is just the enormous majority of us deciding we don't want the default and saying so. Understand that the technology is dangerous. Ask for limits. Refuse to treat the pace as fixed.

Can you and I do anything? Maybe only this, and it might be more than it sounds. We can refuse to call the speed inevitable. Eight millionaires want you to believe it's too late, Harris says, the same way it always feels too late right before someone proves it wasn't. We can be two of the people who slow down long enough to ask the question the whole race is designed to skip.

Chris asked me what my goal of writing about AI was. After reflecting on his question, here's the most honest answer I have. My goal is to invite you to this virtual table and ask the question that could change the most important outcome the human race may ever face.

Why are we driving so fast?

Historically, we don’t discuss safety until harm is done. With stakes this high, I’d like to see us try. It starts by asking why we're driving so recklessly. Then asking why again and again until we identify the root cause.

If we can pull back the symptoms and identify the underlying why, we can design and build sustainable solutions for a brighter future, instead of racing toward our mutually assured destruction.

/Andy