Just to add on “The rise of Grand Seiko has leveled that playing field quite a bit”: there is also Credor. Owned by Seiko and (maybe) not that well known outside Asia
I became interested in Seiko watches after recently buying a Seiko Astron while visiting Japan. It’s amazing how much they shaped the watch industry. With the Astron series they buried a lot of Swiss manufacturers when it was first launched. Also, their “Spring Drive” technology is amazing. It’s basically the perfect hybrid between a mechanical watch movement and a quartz crystal. The movement is mechanical, with the spring releasing energy and powering the quartz crystal (there is no battery), which in turn acts as a “referee”, applying an electromagnetic brake and regulating the mechanical movement.
What if the problem is not that we overestimate LLMs, but that we overestimate intelligence? Or to express the same idea for a more philosophically inclined audience, what if the real mistake isn’t in overestimating LLMs, but in overestimating intelligence itself by imagining it as something more than a web of patterns learned from past experiences and echoed back into the world?
I think AI skeptics have a strong bias to assume that human intelligence fundamentally functions differently from LLMs. They may be correct, but we don't have a strong enough understanding of human cognition to make the claim in as uncertain terms as the skeptical argument is unusually made. The training methods between human learning and machine learning are obviously fundamentally vastly different as are the infrastructure-level mechanics. These elements are likely never going to align, though with time the machine infrastructure may start to increasingly resemble human bio hardware. I bring this up because these known vast differences may account for a significant portion of the differences in expected output from human and machine processing. We don't understand the fundamental conceptual "black box" portions of either form of processing well enough to state definitely what is similar or dissimilar about those hazy areas. Somewhere within that not-well-understood area is what we collectively have vaguely defined "intelligence." But also within that area are all the other aspects that both humans and now machines are quite good at - prediction, fluency, translation. The challenge of lexicon and definition is potentially as difficult a task as is sharpening the focus of our understanding of the hazy black-box portion of both machine processing as well as human processing. Until all those are better defined I don't think we have a good measure for answering the question of machine intelligence either way.
LLM's fail because their input data is limited in dimension compared to humans (text, pictures, audio, video) and because their capacity to rewire their own brains is limited to the transformer architecture.
They pivoted to regular deep learning when Jeff stepped away from the company several years ago. It does not appear they're doing much of brain modeling these days. Last publication was 3 years ago.
reply