I thought about it on Masto in January (the responses are interesting if you want to read), but recently Robin Sloan eludicated it a lot better:

What makes the AI chatbots and agents feel light and clean, here and now in 2026? Is it an innate architectural resistance to advertising, to attention hacks, to adversarial crud? No — it’s that they are simply new! The language models in 2026 are Google in 1999, Twitter in 2009. Their vast conjoined industry of influence hasn’t yet arisen … though it is stirring.

And I believe their architecture makes them more susceptible to adversarial crud, not less. I suppose we’ll see.

It’s interesting and useful to imagine — really visualize — the chatbots and agents in ten years or twenty … barnacled with gunk … locked in a permanent cat-and-mouse game with their adversaries … just as a platform like Google is today. In 2036, you send your AI agent out into the internet, and it returns battered, bedraggled, inexplicably enthusiastic about a bargain flight to Bermuda.

This is no criticism — just an observation about the way things go.

The AI community tends to say “this is the worst this will ever be” in response to criticism, but in a very learned sense, in many aspects it is also the best it will ever be.

Or maybe, to steal words from another person smarter than me, Ted Chiang:

I tend to think that most fears about A.I. are best understood as fears about capitalism. And I think that this is actually true of most fears of technology, too. Most of our fears or anxieties about technology are best understood as fears or anxiety about how capitalism will use technology against us. And technology and capitalism have been so closely intertwined that it’s hard to distinguish the two.

I remember The Master Switch being an excellent book that taught us how to spot and anticipate these patterns. It might be worth a re-read.