Intel’s Q1 2025 earnings press release talked up their new AI-enabled chips. But these are not selling. [Intel] In the earnings call, CFO Dave Zinsner mentioned they had “capacity constraints in In…
A lot of these systems are silly because they don’t have a lot of RAM and things don’t begin to get interesting with LLMs until you can run 70B and above
The Mac Studio has seemed an affordable way to achieve running 200B+ models mainly due to the unified memory architecture (compare getting 512GB of RAM in a Mac Studio to building a machine with enough GPU to get there)
If you look the industry in general is starting to move towards that sort of design now
The framework desktop for instance can be configured with 128GB of RAM ($2k) and should be good for handling 70B models while maintaining something that looks like efficiency.
You will not train, or refine models with these setups (I think you would still benefit from the raw power GPUs offer) but the main sticking point in running local models has been VRAM and how much it costs to get that from AMD / Nvidia
That said, I only care about all of this because I mess around with a lot of RAG things.
I am not a typical consumer
ah yes the only way to make LLMs, a technology built on plagiarism with no known use case, “not silly” is to throw a shitload of money at Apple or framework or whichever vendor decided to sell pickaxes this time for more RAM. yes, very interesting, thank you, fuck off
Built on copyright infringement, not plagarism (the data scientists that built them aren’t going around pretending all the training content was their ideation/creation). There are so many NLP tasks and areas of research that have been wholly obsoleted due to LLMs including sentiment analysis, summarization, translation, NER, data extraction, etc., so I find the “no known use case” to be rather ignorant. The first consumer computers also had tremendous costs. Paradigm shifts in technology are never cheap until scaling.
I think this comment is unfair toward @[email protected] and you should reflect a bit on why that is.
also you’re right, I was unfair towards cubism_pitta. I wasn’t enough of an asshole.
hey fuckers. next time you get the sudden, overwhelming urge to jack off in public about how much money you’re feeding into the machine that does plagiarism and nothing else, keep it the fuck off my instance.
A lot of these systems are silly because they don’t have a lot of RAM and things don’t begin to get interesting with LLMs until you can run 70B and above
The Mac Studio has seemed an affordable way to achieve running 200B+ models mainly due to the unified memory architecture (compare getting 512GB of RAM in a Mac Studio to building a machine with enough GPU to get there)
If you look the industry in general is starting to move towards that sort of design now
https://frame.work/desktop
The framework desktop for instance can be configured with 128GB of RAM ($2k) and should be good for handling 70B models while maintaining something that looks like efficiency.
You will not train, or refine models with these setups (I think you would still benefit from the raw power GPUs offer) but the main sticking point in running local models has been VRAM and how much it costs to get that from AMD / Nvidia
That said, I only care about all of this because I mess around with a lot of RAG things. I am not a typical consumer
ah yes the only way to make LLMs, a technology built on plagiarism with no known use case, “not silly” is to throw a shitload of money at Apple or framework or whichever vendor decided to sell pickaxes this time for more RAM. yes, very interesting, thank you, fuck off
Built on copyright infringement, not plagarism (the data scientists that built them aren’t going around pretending all the training content was their ideation/creation). There are so many NLP tasks and areas of research that have been wholly obsoleted due to LLMs including sentiment analysis, summarization, translation, NER, data extraction, etc., so I find the “no known use case” to be rather ignorant. The first consumer computers also had tremendous costs. Paradigm shifts in technology are never cheap until scaling.
I think this comment is unfair toward @[email protected] and you should reflect a bit on why that is.
also you’re right, I was unfair towards cubism_pitta. I wasn’t enough of an asshole.
hey fuckers. next time you get the sudden, overwhelming urge to jack off in public about how much money you’re feeding into the machine that does plagiarism and nothing else, keep it the fuck off my instance.
soz kid, you’re definitely not tall enough for this ride. maybe go over to the back there and have a lollipop
Removed by mod
Enclosed please find one (1) complimentary ticket to the egress.
LOL. I checked the modlog. Man, the monocle fell off that sealion real quick.
it’s like a magic phrase! remarkably effective
lol
OK, I will reflect on why you think that comment was unfair.
okwhateverdude