It's funny to pick a birth year from our era and think of how history will represent us:
Martha was born in the countryside near Short Pump, in 1993 CE. Her village holds about 300 people. She makes her home in a timber-framed detached house. Martha has neither brothers nor sisters. She eats mostly wheat bread and maize, pork and dairy, and potatoes and garden vegetables, with apples in season, and coffee, soft drinks and snack foods too. She knows her letters. Her height is about 171 cm (5 ft 7 in).
An “Eastern Woodlands” person born 1989 is recorded to have experienced a “volcanic winter” and “dimmed sky” at age 2 as a result of the eruption of Mount Pinatubo.
This is an… interesting contextualization.
I am wondering how many historical situations are sexed up to be something significant, yet were nothing to write home about in the moment. All it took was a historian with an active imagination to bring it to the forefront.
I wonder if that number can be adjusted based on the amount of arable land, or based on the ease of construction (quite the nebulous term here admittedly). The number of mountains presumably makes this hard to compare.
India is 31, Netherlands is more dense than India. Would not have expected that, but then I remember that India has a massive desert, and the Himalayas. So I guess it makes more sense now.
That is an utterly meaningless statistic. Canada, with four times the population, ranks #233, because most of the country is uninhabited / uninhabitable.
Population weighted density is a better metric for this use case. It's more stable than population density when adding large areas of sparsely populated land, because the denser, more highly populated areas are more heavily weighted. It shows, roughly, the density experienced locally by the average person in some region.
The problem is it's difficult to compare across polities because nobody will agree on the right granularity of parcel size to use (and indeed, it is not really obvious what the right granularity is, and choice of parcel size can drastically change the number).
The entire human population can fit within Los Angelas. It’s not a good metric in general. Pressure on public services, resources and housing is far more useful
The entire human population could stand there: each person would have roughly 1.5 to 5.4 square feet of space—less than the size of a single chair. If you actually did get all of humanity in LA it would instantly break out in war.
Usually when something like this is reported, it is because of some other milestone.
Like, they have 6 weeks, on hand, in tanks already delivered.
But, all of the ships in-bound are now done.
After the war started, there was a record number of ships, already filled, already in-transit. But now they have all reached their destinations. So there is no more incoming.
This is awesome! How far away are we from a model of this capability level running at 100 t/s? It's unclear to me if we'll see it from miniaturization first or from hardware gains
One big bottleneck is SRAM cost. Even an 8b model would probably end up being hundreds of dollars to run locally on that kind of hardware. Especially unpalatable if the model quality keeps advancing year-by-year.
> Or we develop a new silicon process that can mimic synaptic weights in biology. Synapses have plasticity.
It's amazing to me that people consider this to be more realistic than FAANG collaborating on a CUDA-killer. I guess Nvidia really does deserve their valuation.
I mean if it was small enough to fit in an iPhone why not? Every year you would fabricate the new chip with the best model. They do it already with the camera pipeline chips.
On smartphones? It’s not worth it to run a model this size on a device like this. A smaller fine-tuned model for specific use cases is not only faster, but possibly more accurate when tuned to specific use cases. All those gigs of unnecessary knowledge are useless to perform tasks usually done on smartphones.
Probably 15 to 20 years, if ever. This phone is only running this model in the technical sense of running, but not in a practical sense. Ignore the 0.4tk/s, that's nothing. What's really makes this example bullshit is the fact that there is no way the phone has a enough ram to hold any reasonable amount of context for that model. Context requirements are not insignificant, and as the context grows, the speed of the output will be even slower.
Realistically you need +300GB/s fast access memory to the accelerator, with enough memory to fully hold at least greater than 4bit quants. That's at least 380GB of memory. You can gimmick a demo like this with an ssd, but the ssd is just not fast enough to meet the minim specs for anything more than showing off a neat trick on twitter.
The only hope for a handheld execution of a practical, and capable AI model is both an algorithmic breakthrough that does way more with less, and custom silicon designed for running that type of model. The transformer architecture is neat, but it's just not up for that task, and I doubt anyone's really going to want to build silicon for it.
> Realistically you need +300GB/s fast access memory to the accelerator, with enough memory to fully hold at least greater than 4bit quants.
The latest M5 MacBook Pro's start at 307 GB/s memory bandwidth, the 32-core GPU M5 Max gets 460 GB/s, and the 40-core M5 Max gets 614 GB/s. The CPU, GPU, and Neural Engine all share the memory.
The A19/A19 Pro in the current iPhone 17 line is essentially the same processor (minus the laptop and desktop features that aren’t needed for a phone), so it would seem we're not that far off from being able to run sophisticated AI models on a phone.
Agree with the first part - but I can run GPT OSS 20b, a highly capable model on my laptop with 32GB of RAM at speeds that for all practical intents is as fast as GPT-5.4 and good enough for 90%+ of non-technical use cases.
As such I can't agree with "The only hope for a handheld execution of a practical, and capable AI model is both an algorithmic breakthrough" - we are much closer than 15/20 years to get these on a phone
With this work you can run a medium-sized model like GPT OSS 20b at native speed even while keeping those 32GB RAM almost fully available for other uses - the model seamlessly starts to slow down as RAM requirements increase elsewhere in the system and the fs cache has to evict more expert layers, and reaches full speed again as the RAM is freed up. It adds a key measure of flexibility to the existing AI local inference picture.
KV-cache is still quite small compared to the weights. It can stay in memory for reasonable context length, or be streamed to storage as a last resort. This actually doesn't impact performance too much, since we were already limited by having to stream in the much larger weights.
A long time. But check out Apollo from Liquid AI, the LFM2 models run pretty fast on a phone and are surprisingly capable. Not as a knowledge database but to help process search results, solve math problems, stuff like that.
It will never be possible on a smart phone. I know that sounds cynical, but there's basically no path to making this possible from an engineering perspective.
Martha was born in the countryside near Short Pump, in 1993 CE. Her village holds about 300 people. She makes her home in a timber-framed detached house. Martha has neither brothers nor sisters. She eats mostly wheat bread and maize, pork and dairy, and potatoes and garden vegetables, with apples in season, and coffee, soft drinks and snack foods too. She knows her letters. Her height is about 171 cm (5 ft 7 in).