I also want it local only because I don’t want X company monitoring my queries, nor to be dependent on yet another cloud based service that can be enshittified at any moment.
Since you can actually run an LLM or a image generation model on midrange GPUs (more VRAM helps, but for my application the 8GB i have work out), i’d say in the current state of affairs its actually pretty robust against enshittification - they don’t even have figured out yet how to stop it to do things they dont want it to do in a guaranteed way. Noone can delete the .GGUF files from my hard drive, and storage is cheap - i own them just like i own my MP3 collection. And that’s the way it should be for far more things.
I also want it local only because I don’t want X company monitoring my queries, nor to be dependent on yet another cloud based service that can be enshittified at any moment.
Good news it comes pre-enshittified
Since you can actually run an LLM or a image generation model on midrange GPUs (more VRAM helps, but for my application the 8GB i have work out), i’d say in the current state of affairs its actually pretty robust against enshittification - they don’t even have figured out yet how to stop it to do things they dont want it to do in a guaranteed way. Noone can delete the .GGUF files from my hard drive, and storage is cheap - i own them just like i own my MP3 collection. And that’s the way it should be for far more things.