I agree, but I also think as AI gets better we're going to see Jevons paradox in full swing, which might delay lower costs. We've seen this with Astra according to Tibo: https://x.com/thsottiaux/status/2097559315150426222

Yeah that's when the site lost me too. I feel like people just tell AI "make the thing" and then get mad when it doesn't match up to their vision that they didn't specify at all.

This is a succinct summary of most the the profession of software engineering. Just replace AI with "programmers".

Yeah I'm surprised they posted a chart, you would think they would keep specifics like that hidden until they're closer to launch

The chart is as non-specific as could be. It improved in some very vague metric by some amount at different (increasing) levels of training.

That's fair, but at least the chart has an axis. :) Since openai just released astra, I was more surprised that they would publicly show any gap to their (presumably SOTA) internal model.

Isn't the y axis just what portion of the open problems it could solve? The axis is unlabelled though, I'll give you that

The x-axis label of the chart is test-time compute. Doesn't this relate to inference ("thinking level") instead of training?