• 0 Posts
  • 12 Comments
Joined 1 year ago
cake
Cake day: February 20th, 2024

help-circle

  • If I use a password manager with long random passwords, and use 2FAS to generate those 6-digit two factor authentication codes whenever possible (as opposed to SMS/email 2FA), is there any advantage?

    Is it just that you don’t actually have to type anything, just press “I approve” on your phone after entering your username?

    Or is it more just designed to improve security for people like my family members who use the same ~10 digit passwords for everything?



  • Yes. The Llama 70B derived models, as well as Mixtral 8x7B and the new Mistral Medium 70B are competitive with ChatGPT 3.5. Most of them can do 16,000 token context similar to ChatGPT as well.

    You only NEED 40GB of free RAM to run them at decent quality, but it’s slow.

    With a 24GB GPU like a 3090 or 4090 you can run them at a reasonable speed with partial GPU offload. About 1-2 words per second. I run 70Bs in this manner on my computer.

    With two 24GB GPUs you can run them very fast, like ChatGPT.


    There’s of course a whole world in between as well, but those are the rough hardware requirements to match ChatGPT in a self-hosted sort of way. There’s also a new thing people are doing where they add layers from one model onto another one, like a merge but keeping >50% of the original layers from each model. “Goliath 120B” and the like, which is made from 2 different 70Bs. They’re even better but it’s a bit beyond reasonable consumer hardware for now.






  • GoogleyWoog@lemmy.mltoSelfhosted@lemmy.worldStarting from zero
    link
    fedilink
    English
    arrow-up
    1
    ·
    edit-2
    1 year ago

    Thanks. Yeah I already leave it on most of the time for media streaming and torrents and stuff. Electricity here is only about $0.08 USD/kWh. I of course don’t support unnecessary waste but it doesn’t seem to make a big difference.

    Edit: for fun I did electricity cost calculation. If we call it 50W idle power usage, running it 24/7 is only $3/month.



  • If this sort of thing is light enough to run on a Raspberry Pi or old laptop, is there anything wrong with just running it in the background on a modern(ish) computer? My desktop is a 5900x with 64GB of RAM. I already run Jellyfin, Sunshine, and a few other things in the background but kind of want to add a web server and mess around with some other stuff.

    My main concern would be security I suppose if I’m hosting a web server on the same computer I store all my family backups and stuff. Would using virtual machines solve that?