Can you describe a method of slowing frontier AI development that you yourself would not classify as monopoly-seeking by US AI companies or power-seeking by governments and technocrats?
Limiting the development of powerful AI requires restricting free enterprise in some way, backed by the threat of force; hence, there will always be a cynical framing available.
I believe this is a misunderstanding based on inadequate coordinate systems, and that an astronaut would fall through the event horizon, die, and reach the singularity in finite time.
Outside of the most niche domains like frontier AI development, are there software engineering questions that Sol 5.6/Fable 5 can't answer better than the median software engineer if explicitly asked? It strikes me that models already have superhuman software architecture capabilities, they are simply underelicited.
Furthermore the AI companies are spending gobs of money and compute to address this—harness development, orchestration/agent coordination, computer use are all designed to lower the skill level required to get the best results out of a given model.
Soon the models themselves will be more persistent, more observant, and more skilled than any of us at driving models to obtain the "right" solution in a given SWE context.
In terms of getting a the consensus answer to a specific question, they're likely better than a median software engineer at most questions.
Where they still fall massively short, even compared to a straight out of college junior developer, is figuring out what the correct question to ask is.
This has led to situations like Fable 5 spending six days debugging a deadlock because it didn't think to add any observability to detect when the job stalls, and would wait until the job timed out each time. Any junior developer would have fixed it in a few mins, but I left Fable running out of curiosity to see how long it'd take.
This may be true now but isn’t the trend clear? GPT-2 couldn’t write a coherent function. Now we have the SWE oracle that can autonomously navigate your codebase, write code at a superhuman level, and give the correct answer to most engineering questions.
If anything I feel I am understating the capability of current models here. What is stopping the next generation of models from closing the feedback loop further?
> Outside of the most niche domains like frontier AI development, are there software engineering questions that Sol 5.6/Fable 5 can't answer better than the median software engineer if explicitly asked?
"Median" is kinda hard to judge given how rapidly the number of software engineers has been growing, how imprecise our within-industry skill assessments are, and also how fast everything within software was shifting even before agentic coding. Two examples: (1) an old estimate I heard was that 50% of the skills we would put on our CVs would be obsolete every few years, but we'd never know which ones; (2) The reason for the creation of FizzBuzz as an interview question (far too many people are *that* bad).
For the LLMs themselves: the METR time horizons estimates are based on an expert developer's wall-clock times, and the graph on their website only goes as far as Mythos preview which is kinda about this level. Mythos preview shows only 80% success at tasks that would take a human 3h6m on average (95% confidence interval 1h37m-6h39m), which is evidence that yes, there is plenty that the models can't do. I naturally break down larger tasks to smaller ones, the agentic coding models I've used also attempt to do this, but seem to be worse at it than I am.
> It strikes me that models already have superhuman software architecture capabilities, they are simply underelicited.
While they can do better when elicited correctly (and the same is true for humans), they are too "lazy" by themselves to attempt self-elicitation.
The second part, just like the first, is also true for humans; but LLMs are so much cheaper than humans that this is a disappointment for the LLM to stop early and say "[Thought for 35m 24s] Here's your MVP, I've not done subtasks 35 onwards", even though it's an advantage for the humans to say "[Invoice for a 2 week sprint with a team of 5 developers and their manager for a total cost of $23,000] Here's your MVP, we can discuss which of subtasks 35 onwards you want in the next sprint".
I'm not sure where I'd rate LLMs on a strict 0-10 scale from "hack" to "formal methods followed by end-to-end testing", but the examples given by the research companies themselves suggest that even they are only at the "this is starting to get interesting" level for both architecture and their own idea of what "I have now completed this to a good quality" looks like, not at the "actually good" level for complete big projects.
> Furthermore the AI companies are spending gobs of money and compute to address this—harness development, orchestration/agent coordination, computer use are all designed to lower the skill level required to get the best results out of a given model.
The observation is that money and compute spent are not strongly correlated with the general quality of the models. If the correlation was strong then Meta would be near the top, while the multitude of open-weight Chinese models that keep popping to the top of leaderboards every couple of months would be jokes. Grok would have been better from the start and not suddenly gotten good around when they bought up the much smaller and lower-budget Cursor.
> Soon the models themselves will be more persistent, more observant, and more skilled than any of us at driving models to obtain the "right" solution in a given SWE context.
My experience has been that predictions of "soon" fare about as poorly as predictions of "never" in the context of AI. Regarding the failures of "soon" predictions, I do not know the degree to which this is company leadership BSing how good their stuff is, vs. Goodhart's law where they're fooled by the difference between their metrics and reality.
Whenever someone tell me they can be reduced to an LLM, I believe them. I see you and I believe you that you are qualitatively the same as a language model.
It also creates and updates models, uses those models to make predictions, guides the stochastic generation by comparing the output to those models and reworks them in real time.
It also creates and updates models, uses those models to make predictions, and guides the stochastic generation by comparing the output to those models and reworks them in real time.
“In recent years, as artificial intelligence has advanced, Flock has added the ability to search for cars based on their make, type, color, or unique details, such as bumper stickers or visible damage.” So TOTP license plates is not enough.
TOTP license plates would be totally rad to see imagined in a badass dystopian sci-fi film. I'm imagining something along the lines of Robocop or Judge Dredd.
IRL we will probably need more mature solutions - legislation, regulation etc. At this point we know the tech can't really be contained. It'll be all-seeing, all-knowing and super cheap. Just as the source/link describes it. So the only way is to have actual adults working in law enforcement.
Right, I feel like I'm taking crazy pills reading all the comments taking this article seriously.
I think AI has broken HN. The readership is so head-in-the-sand about AI that they wouldn't recognize 2000 words of Claudeslop if it genuinely slapped them in the face
The alternative explanation (the top commenters actually read it and somehow didn't recognize it's obviously written with AI) is just too bleak and I choose to not believe that.
Ed Zitron has predicted 16 of the last 0 AI bubble bursts...
But more seriously, this has to do with DeepMind falling way behind the frontier in intelligence and cost per task. There's basically no reason to use a Gemini model today.
reply