
The biggest risk in AI is centralized power. As much as the EAs and rationalists want to convince everyone that an AI can convert all of us into paperclips, that we’re creating something that will treat us like ants or 7-year-old kids, that’s just thinkism, as Kevin Kelly says. The real risk is centralizing the power of such a tool to either a few companies or governments.
There are two ways we can end up with centralized control: governments or companies. On the government front, it’s not a win yet. All kinds of forces are trying to get AI regulation passed: politicians looking for more power, companies looking for regulatory capture, and even the Pope now with his latest encyclical. I think he nailed the centralization risk in companies without realizing there’s the same one when you regulate them. Actually, government centralization is worse, at least companies have the government watching over them, but who watches the watchers?
On the other front, things are looking as good as we could hope. We have all kinds of models and companies competing: SOTA models pushing the limits every other week, open weights models letting users run private and local model AI without asking for permission, American models, Chinese models, we even have a European one. Intelligence is becoming more and more like a commodity, and the app layer seems to be what’s important.
As long as models are swappable and users have the possibility to exit and switch, that risk is completely mitigated. That kind of environment leads to competition, and competition forces companies to behave well. If they misbehave, they start to lose users fast.
So now we only need to worry about the government side, and even there things are looking better than they could. Regulation happens country by country. If one country decides to lock everything down, the rest of the world still has access to unregulated models. That creates a natural incentive against over-regulation, no country wants to handicap itself while everyone else races ahead. This is already happening; China has, ironically, open sourced its models to try and damage US companies’ revenue, giving access to unrestricted models to users around the world.
All the right incentives are starting to align. The future of AI looks less like 1984 and more like the internet: messy, distributed, and impossible to control. The kind of environment that lets a thousand models bloom.