Lol, a couple of years go, grown men were getting into fist-fights over toilet paper and it's looking like they are about to wage war over a tank of juice.
Suppose I make a paper wallet on an offline PC, write down the address and discard both keys. If I now send some BTC to this address, how does the client figure out the public key?
Cerebras have special techniques to work around etching errors / bad cores on their wafers. This is possible since their wafers are effectively hundreds of identical copies of redundant cores. Can't do that for a globally unique model.
Etching failure in that situation would be like brain-damage in a human, all sorts of weird effects would start appearing.
There's several ways to engineer around that as the errors are detectable. There's a big literature on how to trade off speed or transistors for error correction. [1]
(Is Cerebras doing something novel? CPUs and memory blocks have been doing those things for a long time too, since the error rate is otherwise too high for normal size chips as well)
A few hundred bad bits/transistors in a trillion+ parameter model would compromise its abilities not one iota...the models are inherently lossy and resistant to "brain damage"...
Off the top of my head, events in popular science fiction that have "already occurred":
First Laughing Man Incident - Ghost in the Shell: Stand Alone Complex (2024)
Third Impact - Neon Genesis Evangelion (2015)
Fourth Corporate War - Cyberpunk (2022)
Geisel Library Walk - Rainbows End (2025)
2020 Tokyo Olympics - Akira (2020)
Very good timing, not for walking houses and sharding of consensus reality with ubiquitous AR, but for physical destruction of books to speed up scanning.
Domingo Gonsales is flown to the moon by tying himself to wild swans, and meets Lunar Christians - The Man in the Moone by Francis Godwin (1601)
Manned mission to the Moon by giant gun - Jules Verne (1865 ish).
Manned trip around Saturn by comet - Jules Verne (1877).
Time traveller sets off to the future - H.G. Wells (1894)
Martian invasion of Earth - War of the Worlds by H.G. Wells (1905)
Global telephone network reaches as many connections as a human brain and wakes up - Dial 'F' for Frankenstein, Arthur C Clarke (December 1, 1975)
Birth of Dr Susan Calvin, founding of US Robotics & Mechanical Men, development of 'Positronic Brain' - Asimov's robot stories (1982 and after)
British space station under construction - The Outward Urge by John Wyndham (1994)
Portal to parallel universes - Sliders (1994)
Doctor Samuel Beckett steps into the Quantum Leap Accelerator and vanishes - Quantum Leap (1995)
Skynet becomes self-aware - Terminator 2 Judgement Day (August 29, 1997. at 02:14 am Eastern Time)
HAL 9000 AI computer operational (1997), first human voyage to Jupiter system (2001), first landing on Europa and alien life discovered (2010) - Arthur C. Clarke's 2001 A Space Odyssey
Moon leaves Earth Orbit after nuclear waste explosion, taking human-occupied Lunar colony with it - Space:1999 (13 September 1999)
Efficient international flight, using Zeppelins and blimps - Rudyard Kipling (2000)
Global climate disaster - The Day After Tomorrow (2004)
Replicants - Bladerunner (2016)
Keanu Reeves smuggles digital information in his brain - Johnny Mnemonic (2021)
Arnold Schwarzenegger kills to survive in a typical American TV show - The Running Man (2025)
Woman relifed into robot, rich people discover workers' rights - Fritz Lang's Metropolis (2026)
First colony mission to Mars, 100 colonists on the largest spaceship ever built - Kim Stanley Robinson's Red Mars (2026)
Tibetan monks finish listing all the names of God, stars start going out - The Nine Billion Names of God, Arthur C. Clarke (when computers were rentable and could print 1000 names per second)
I think so, specifically lossy compression though.
A modern version of the book would include an extra section in the 'Lossy compression' chapter - 'Text' (alongside Images/Video/Audio) that would discuss LLM's.
An LLM can give you a probability distribution for the next token. You can pair that with arithmetic coding to get a lossless compression/decompression algorithm. See https://en.wikipedia.org/wiki/Arithmetic_coding
In the way that you say, you can do lossless data compression, but then the LLM is used in a very distinct way than it is used in applications like chat or coding assistance.
In the latter applications, you do queries which aim to extract information from the training data set, but which may return hallucinated content instead of correct content.
If you use an LLM just to provide an estimation for the frequencies of tokens in an input data stream, and then you use the estimated frequencies to encode the input data, then you do not care about which were the tokens predicted by the LLM, because they are not used. The worst effect of any wrong predictions by the LLM is a slightly worse data compression ratio than the optimum.
When it is said that LLMs do a lossy data compression, that refers to the compression from the training data set to sequences of output tokens.
> If you use an LLM just to provide an estimation for the frequencies of tokens in an input data stream, [...]
Why would you use an LLM for that? The whole point is to encode contextual probabilities. So basically: given this prefix of text, what's are the probabilities for next tokens? You can use this conditional probability distribution to sample from to create plausible text, or you can use it for lossless compression. The math is very similar.
LLM's seem to be the weird interesting outcome of applying lossy (de)compression concepts to text instead of the audio/image/video domains where they have traditionally been used.
reply