Don't know if you're referring to the headline or the body (which is paywalled). The current headline reads "OpenAI says it has overtaken Anthropic with its latest AI model". Which makes me wonder whether FT itself changed a headline along the lines of what you wrote in the past few minutes?
Apparently Greg Brockman said that as far as he's concerned it may be AGI, or something along those lines.
OTOH OpenAI have their own meaningless definition of AGI as "able to do most commercially valuable work" or somesuch, which I'm sure is not true, and is also not what I'd call AGI.
If it can't learn for itself, then it's certainly not AGI.
The biggest significance of anyone at OpenAI calling it AGI is their contract with Microsoft, giving MSFT access to all their IP, is based around them having achieved AGI, but presumably they can't just declare that unilaterally.
Maybe they see getting out of the Microsoft contract as necessary to IPO.
>OTOH OpenAI have their own meaningless definition of AGI as "able to do most commercially valuable work" or somesuch, which I'm sure is not true, and is also not what I'd call AGI.
It's not meaningless. Specifically it is - "highly autonomous systems that outperform humans at most economically valuable work". In fact, it's one of the most meaningful definitions out there.
>If it can't learn for itself, then it's certainly not AGI.
Now this is meaningless.
>The biggest significance of anyone at OpenAI calling it AGI is their contract with Microsoft, giving MSFT access to all their IP, is based around them having achieved AGI, but presumably they can't just declare that unilaterally.
They can't declare it unilaterally, and those deals have changed significantly. Microsoft’s license to OpenAI models and products runs through 2032, and explicitly includes post-AGI models.
> It's not meaningless. In fact, it's one of the more meaningful definitions out there.
Well, it's commercially meaningful, but it's not meaningful it terms of how close we are to achieving human intelligence.
But which one you care about is up to you. If you are all about the money and don't care about human intelligence then for sure go with OpenAI's definition.
It's a lot more than just commercially meaningful. Even if you don't care about money, "highly autonomous systems that outperform humans at most economically valuable work" has a lot more weight on "how close we are to achieving human intelligence" than most of the 'AGI' thresholds I've seen yet.
Traditionally, AGI means being capable of learning everything (not necessarily at once) that can be learned by the same AI agent. It differs from the commercially meaningful definition in that a standard specific-purpose-built AI can still do most tasks if you give it enough specific purposes, but it will be just as incapable of human intelligence as ever.
No - the term "AGI" was really coined to distinguish general intelligence from narrow intelligence(s). There's an obscure earlier usage, but it basically became mainstream as the title of a book edited by Ben Gortzel who had in turn got it from Shane Legg.
Shane Legg would a few years later go on to co-found DeepMind, with creating AGI as their declared mission. Legg's personal definition of AGI is not just generalist AI, but specifically human-level generalist.
Right now we have just massively jagged intelligence that performs stunning feats in math and single-shotting three.js games, and falls flat on it's face in many every day scenarios.
All intelligence is jagged. Human intelligence is jagged, so is Cetacean intelligence, so is Ape intelligence, so is Corvid intelligence. You could re-orient that statement for any species of intelligence relative to the other, or even individuals within a species. The big problem here is people being so convinced of their specialness they'll just keep sticking their heads in the sand, until we essentialy create God, and by then none of this useless pedantry will even matter. If you can't see we've already learnt to walk then there's nothing more to discuss here.
I'm not sure it's accurate to call human intelligence jagged - it seems (perhaps largely thanks to language) that we're able to make progress on anything we set our minds to, and of course we're now in process of building AI that will eventually be able to do things that we ourselves couldn't, whether due to computational demands, or memory capacity, or having sensory inputs that we don't have, etc. But at the end of the day, AI is a tool we've built, so maybe it will make us less jagged, and certainly more capable.
In any case, human intelligence as a goal is special since we live in a human world, and one of the prime goals of AI is to be able to do human jobs. If a human can do something and AI can't then that is a limitation, while if a human can't and AI can't either, that doesn't matter for the time being.
Yeah, I shouldn't have said AI can't walk yet, but most of what it can do is just due to scale - we're still just building big transformers, basically using the same 10 year old architecture that accidentally set us off on this path. I'm not sure we're really progressing towards human/animal type intelligence, just learning how much of our own capability can be realized by automating language.
I'm not sure it's meaningful to compare across different types of intelligence--but I don't think human intelligence is so special that we can pretend it's much less jagged than all other animals for sure. Our scales of models are also not far off biological, probably less efficient but not like astronomically so i would guess.
But anyhow I agree that inference-time learning doesn't exist and it's a big issue still I think.
There seems to be very few cases where ICL is really doing something that can be considered as learning (albeit ephemeral) rather than just utilizing in-context data via induction heads.
Yes, I'm aware of the linear regression example, and a few others, but these appear to just be specific capabilities that were learnt during pre-training, presumably pursuant to reducing errors on similar-but-different training samples, not any kind of generic run-time learning capability.
> The biggest significance of anyone at OpenAI calling it AGI is their contract with Microsoft, giving MSFT access to all their IP, is based around them having achieved AGI, but presumably they can't just declare that unilaterally.
>
>Maybe they see getting out of the Microsoft contract as necessary to IPO.
I don’t have a link handy, but they already loosened up that contract significantly earlier this year, surely because they planned to start claiming AGI to pump their IPO and didn’t want to be beholden to all of those commitments.
>If it can't learn for itself, then it's certainly not AGI.
What does this even mean, exactly? For example if it can filter new information to put into the next version of itself does that qualify? If not, explain exactly why that's the case.
Ask 100 people what their definition of AGI is and you'll probably get 100 different answers.
My definition is closer to DeepMind's "can do any [computer-based] task that a human can do", and certainly a human intern can learn on the job and do better on day #2 than day #1.
Learning is basically THE hallmark of intelligence. Being able to learn from experience and use that to do better in the future.
If you could somehow take everything the LLM learnt today and "put it into" an update for tomorrow that would be better than nothing, but the intern on the job is probably learning dozens of things per day that stack upon each other, so that's going to be a S-L-O-W learning AI intern !
Leaving aside the technical issue how you could store a days learning, or what that would even mean, you've also got the privacy and merge issues if this were to be done in the cloud without every customer having their own personalized model.
For real continual learning we need to get past gradient descent-based batch training and develop a new learning algorithm so the model itself learns incrementally as it runs (animal-like predict, observe, learn cycle), rather than being reliant on some external alien to come in, equipped with it's entire learning curriculum, and program its weights.
OK, let's say we get continuous learning today, what does tomorrow look like?
I won't argue that AI can do some learning as context sizes are still terribly small and expensive to iterate.
What does the world like in a week? A month? A year?
How long before these models drift to their own languages? To their own set of morals? To their own alignment?
None of these questions are answered and I'd rather stay at AGI-lite until they are as having a billion agents going off in their own directions seems like a recipe for disaster.
With current models and their general knowledge self learning just seems like you'd have a few ASIs crop up really quick.
Yeah, I'm fascinated by AGI, but not very enthusiastic about it. I've yet to see anyone, notably not even the people trying to build it, articulate a future where it seems to be a net positive for society.
If it's a choice between everyone living on food stamps or reducing the infinite prime gap to 140, then I guess I'll be happy with a prime gap of whatever it was yesterday.
If/when we do eventually build a more animal-like true human level AGI - build a brain, not just a language model, then potentially we'd be able to build in some of the safeguards that millions of years of co-evolution have built into ourselves, but I expect that is still decades away, and then it'd be capable of doing even more jobs!
The best AI outcome I could wish for is where it is extremely heavily regulated, and AI replacing any human job paying under $1M/yr is banned.
I also think it's moving the goalpost significantly. If you asked me what AGI looks like in 2024, I would have said "smarter on tasks that can be done via text than the average random human you'd meet in a NY bodega", and by that standard, it's long in the rear view mirror.
Now I think people are asking a different question, which is "better than any human at any task that can be done via text", which imo is superhuman, not artificial general intelligence.
It's not moving the goalposts - it's just having a different goal in the first place. This has been Shane Legg and Demis Hassabis' definition of AGI since DeepMind was founded, or before (way before OpenAI appeared), and also for me the goal of AI has always been human level (really a human brain, complete with emotions, etc, but we can start with basic learning/prediction).
I guess I fundamentally disagree, because on general knowledge they beat any human, and on processing speed they beat any human, and on problem solving likewise. I just don't see a domain besides "retain change over time" that they fail on, and that is something you can do in the system rather than the model - coding agents can and do build their own memory system and use it, like an amnestic human writing notes.
I think what I'm objecting to is the idea that if it can't do every task that every human can do better than any human, it's not AGI - I think we have to allow that this is a fundamentally different form of cognition, so requiring a 100% match feels uncharitable. Every human has cognitive gaps that AI doesn't have.
I'm not sure there is anything to disagree on here - just different definitions of what we personally associate the term "AGI" with. There is no magic to the name, and however you choose to define it there will be less and more capable levels of AI that precede and follow it.
Separate from what you choose to call any given set of AI capabilities, learning is generally more than retaining change over time. Learning isn't the same as memorization - it's the difference between memorizing a stack of how-to books and going out and practicing the skill for yourself, and all that entails.
A learnt skill needs to be internalized such that in the future it's what is driving behavior, which means it needs to be in the weights. Imagine a robot trying to learn to play tennis where rather than updating it's connections between perception and action it's trying to store everything as a bunch of notes!
It's not just physical tasks where this applies - the same would be true to trying to teach a multimodal model to recognize different types of mushrooms from photos, or even learning a new language. If a system can truly learn for itself then it shouldn't need to be pre-trained with language, but rather should be able to learn by immersion and practice the same way we do, and the same way a dog learns to follow commands. You can't do this by leaving notes to yourself - the learning mechanism needs to be animal-like and update the systems weights in response to prediction failure - a runtime version of the way SGD-based pre-training updates weights in response to prediction errors.
> If a system can truly learn for itself then it shouldn't need to be pre-trained with language, but rather should be able to learn by immersion and practice the same way we do, and the same way a dog learns to follow commands. You can't do this by leaving notes to yourself - the learning mechanism needs to be animal-like and update the systems weights in response to prediction failure - a runtime version of the way SGD-based pre-training updates weights in response to prediction errors
I am not sure why any of that is necessarily true. I think you are crediting the human brain with a lot more than it actually does - memory is something in the direction of a fine tune on existing neural hardware, it's really not anything special, and I absolutely think that "just taking notes", when taken to a sufficient fidelity, is enough to pass that bar.
You can't take notes before you've learned language, so if that's how you propose to learn then you are doomed to build something with less learning capability than an human brain.
Maybe you don't care - if you just want to push to see how far an LLM can go that's fine. but it's not going to be superhuman-level, or even human-level, if it doesn't have the learning ability of a human. If it relies on language and notes to itself then it is just an LLM.
Okay, nowhere did I say it would be before they learned language. I said we could use models that can trivially build notetaking systems to auto-improve themselves by taking, refining, and updating notes as a memory system to exceed the x token context threshold.
> but it's not going to be superhuman-level, or even human-level, if it doesn't have the learning ability of a human. If it relies on language and notes to itself then it is just an LLM.
Why do you think that? You do a lot of assuming a thing to be true without any clear reasoning or evidence. I'm talking about a system, including a LLM, that acts in a way that is equivalent to or better than an average human at a large range of tasks, such that is generally capable of learning and problem solving similar to the degree of success humans have. All these specific criteria about what is "actually human" is not the point. We know how to make more humans, I'm talking about a form of intelligence that we are creating that is genuinely alien, so holding to "well it doesn't work the same as a human" is an impossible bar. If you reverse the situation, humans are by no means LLM-equivalent intelligences - perfect memory of long form content, rapid calculation, instantaneous tool calling - a hypothetical silicon based lifeform would dismiss us as "really not intelligent at all, they can't even do multiple floating point calculations per second in their head", and I think that would be similarly silly.
OK, so we're basically talking about different things. I'm talking about building something that has animal/human intelligence and learning capability, and would be able to first learn language for itself (no pre-trainiing), and then proceed from there.
You're talking about building an "alien" intelligence, apparently something based on an LLM, where language is baked in from the start and can therefore be used as the basis of some type of learning (or at least memorization).
Sure an LLM can be super-human at specifc things like math where the traditional computer strengths of compute, memory, etc apply, but in the context of AGI (generality, not a bag of narrow intelligences) you can't call something super-human if it can't at least also do the majority of things that a human can do, and not having learning ability as powerful as a human seems to me to be a pretty massive omission.
If WE didn't have ability to learn language, then we'd not be here talking about it, and there would be no LLMs. Having a fundamental ability to learn, isn't just useful to learn language from scratch, or for learning all the non-linguistic skills a human is capable of, but would also support an AI that can learn things that we cannot (e.g. could give it additional "senses" like global pressure/temperature inputs so it could "see" the weather and learn global patterns).
The author is a superb writer. Actually made me lol:
"I have a problem with artists doing commerce under the guise of art. I listen, I read, and I watch because I want to inhabit, even if just for a moment, the mind of another human. I want to feel what it’s like to be them, and in so doing, I want to better understand what it’s like to be me. But if I journey to the center of someone’s psyche and all I find there is a billboard for Pizza Hut, I’m turning around. If your art is just one node in your business empire, if your albums are merely commercials for your cologne, if you’re trying to turn your first billion into your second billion, you are no longer an artist at all. You are a credit default swap with a discography attached."
Not long ago I was looking at my favorite games of decades past. Unreal Tournament figured very prominently, made of course by Epic. So I wondered: why did they stop making Unreal games? I looked at their game chronology. On one hand, they made Gears of War, an Xbox exclusive that never interested me. And the other one? Oh, right: Fortnite. That's where Unreal Tournament went. They made tons of money for sure. But no company, including Epic, has made a competitive FPS + CTF game as solid as UT, UT2003, or UT2004 since that era
They had Unreal Tournament 4 in development around 2018 but it never gained much traction in the pre-alpha phase. Once Fortnite blew up they seemed to just focus on that and their app store.
Halo Infinite is the closest I've gotten to the UT feeling nowadays. Simple arena, equal playing field, drop in drop out, tools-not-loadouts design. It's a shame how a variable and strong design gets put off into the corner to wither.
I wanted to play UT2004 for some nostalgia recently. Turns out even though I own it on the Epic game store, I can't play it because Epic removed it from the Epic game store.
They removed all UT games from online stores and added UT 3X which is a free version of UT3 with Epic Online Services baked instead of original ones.
The only way one could legally get UT99 is to buy physical. And it's been like that for many years prior to an event above, which also disabled the server browser after 22 years of running intact.
Same reason valve doesn't make games anymore. They followed the money and licensing their engine did a lot more than making games. Any games they made were to showcase the engine.
They just happened to hit the goldmine with Fortnite.
I think the problem is also that there are many FPS multiplayer CTF games even if they are not all great, they all compete for attention in a crowded market. Destiny, Call of Duty and all their variants.
most likely, there seems there are plenty of devs from nearly all major tech companies on HN, they often don't chime in as much anymore when it comes to problems, I've wondered if they get some kind of guidance on not commenting on "problems".
The general guidance is likely what I was told when I worked at Apple: essentially, as an employee, people will read what you write as though you are repenting Apple whether you are or are not.
So in short, I kept my mouth shut. I assumed I would lose my job if my public comment reached the right people.
To this day, even retired, I send bug reports to co-workers I know that are still at Apple. (I've sent a few image files that were problematic to the top engineer on the ImageIO team for example. I worked with him for over two decades before I retired.)
Apple is a very different place than it was when I started in 1995. Over the decades since I started, I have seen numerous changes I dislike. Sadly many of the changes were seen across the whole industry though so I would be no better off anywhere else.
I'm happy to have retired though. The industry lost a lot of what used to be fun.
Half the point of "AI" is to squeeze the labor market. This is why you don't see people chiming in. It's a nearly fully corrupt and monopolized system.
Azure and felt overwhelmed? As a student or first- time user to cloud computing, I've been there too. The idea of creating a chatbot or search app using GPT sounds exciting, but the process of setting up everything right from the vector database, provisioning OpenAl models, to integrating them,
Feel free to create an alternative. Keep in mind it's completely illegal and you will get the book thrown at you if you are caught. You will also end up using your captcha page to DDOS people who are trying to unmask you.
reply