I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
They’re PR for the AI investment hype cycle. We have hard mathematical limits baked into the structure or transformer models (a nice clean line forms on a logarithmic chart no model has ever crossed when you plot out exponential increases in training costs, resources, and time, against capabilities)
So a “self improving” model would show up as a model that takes 8.5X as much compute, electricity, water, and time as the model before it in order to get 1.17X the gain or whatever…
We’ve been capped out on easy gains for 2 years now.
They’re PR for the AI investment hype cycle. We have hard mathematical limits baked into the structure or transformer models (a nice clean line forms on a logarithmic chart no model has ever crossed when you plot out exponential increases in training costs, resources, and time, against capabilities)
So a “self improving” model would show up as a model that takes 8.5X as much compute, electricity, water, and time as the model before it in order to get 1.17X the gain or whatever…
We’ve been capped out on easy gains for 2 years now.