I worked very briefly at Baseten, and I can say that it was a perpetual annoyance (from an engineering perspective) that customers would complain about issues with their models but we couldn't actually see the inputs/outputs. I don't know about the other providers, but at Baseten they literally weren't stored anywhere.
It's a true belief, but that doesn't mean it's valid or correct. Many doomsday cults had true believers, and were able to retcon their beliefs when the predicted day came and went. AI doomers have the luxury of not having committed to a specific date or method (but see https://x.com/archerships/status/2095971206847738362)
In a moment of weakness, I joined a high-IQ group (Triple Nine Society). The whole mailing list was just people having the same political arguments that were on the rest of social media, just with a bigger vocabulary.
Put an airtag into literally every bag you check. Helps you figure out not just whether they made it on the flight, but also if they ended up at some baggage claim on the other end of the airport instead of the one they were supposed to be at
but the temperature? that high heat about is directly from the shitcasting endlessly-complaining right. the right that actively seeks to prevent live and let live, that aggressively is trying to turn everything it can into a battleground. there's not two sides about any of this, it's just endless dark triad victimhood signalling, to rile people up. from the worst. some of that is on the left too, but at least they represent actual victims, and aren't actively trying to promote vice as the victim. https://gwern.net/doc/psychology/personality/psychopathy/202...
the monsters need to turn down the temperature and stop being fucking pathetic loser babies. grow the fuck up, get on with it, you shits. that obviously isn't working so society honestly deserves more active defenses: we need ways to shut down the monsters who can cause stinks, lay shit, flood the zone, make battles out of anything, especially those situations in which yes, some people actually did finally stop being actively systematically hurt, and no, that's not actually a problem, you absolute wankers.
I recommend reading some foundational materials on federated learning and distributed inference. The fundamental issues are (1) compute (2) latency and (3) security.
On the compute side, you can't load most individual experts into even high-end consumer-grade cards (e.g. RTX 5090) because the weights are simply too big.
On the latency side, you need to ship the activations and wait for the somewhat underpowered cards to actually do the matrix multiplications, then send the results back. This means each token takes hundreds of milliseconds or even more, which is borderline unusable.
Finally if you're running a P2P network, the only way to verify that peers aren't lying to you is to run duplicate calculations - I don't see how SHA-256 or signed model state help here unless I'm missing something (What are you calculating the hash of? What exactly is signing attesting to?).
This is one of the few places where borrowing ideas from cryptocurrencies actually makes sense - you could imagine a system where peers place bonds and forfeit them if they lie about calculations. You can look at projects like Bittensor and others for how these kind of things are currently implemented.
Thanks for the comment. Extremely clear and detailed. To write this project, I researched a lot about methods and how peers should work compared to a server, as well as security concepts, which certainly have more to add. The goal for this type of project was to move from a single-machine Colibri to multiple-machine Lumabri on a LAN to a large number of machines working together in a Napster-style P2P Lumabri. For latency and other issues, we are studying every type of method that can improve it, and we are also writing and testing other things on our test server. Thank you very much, we will continue to improve.
Didn't sfcompute launch with a vision of something like this and then cancel it because CFTC decided it was running an unregulated futures exchange? How are you getting around regulatory issues like this?
Good question. this is something we built around from day one. Users pay 100% up front for specific nodes on the exact weeks they want. No leverage, no margin, no cash settlement. This is honestly not that much different from you booking a hotel.
So yes, you do get cash. when i said cash settlement, i was referring to how the contract itself ends, unrelated whether you can sell it. A cash-settled contract means it never delivers anything and at expiry it just pays out the difference between your price and an index price, which makes it a pure price bet.
on computable, whoever holds the reservation when the week arrives gets the actual nodes when they redeem for usage. so selling your reservation to someone else beforehand is no different from reselling a concert ticket. and when you sell, the buyer takes over the claim, you get their money, and the contract still ends with someone getting the GPUs.
After being in the workforce for decades, this whole issue is just so incomprehensible to me.
I went an ungrad school that was top-5 in engineering. But my experience - and in the experience of other people I've talked to - formal undergrad education was, and always has been, a farce. At best, you learn through working on projects that are meaningful to you and learn "how to be an adult" (and later, you learn how to manage the enormous financial debt you acquired). But more typically, it's pure credentialism - no one cares what your grades were, only what school you graduated from.
The amount of actual learning that goes on from classes is minimal, but somehow we can't shift the overton window away from this silly game of grades that don't measure anything meaningful.
After graduating, I've was asked about my grades exactly twice in my life -- once when I applied to a master's program, and at one job interview (the company had a policy of asking about GPA for anyone who graduated less than 10 years ago).
I'm pro-education but anti-school, and all this nonsense makes me this way even more.
reply