RandAlThor@lemmy.ca to Technology@lemmy.worldEnglish · 17 hours agoElon Musk’s Grok Goes Haywire, Boasts About Billionaire’s Pee-Drinking Skills and ‘Blowjob Prowess’www.mediaite.comexternal-linkmessage-square118fedilinkarrow-up1774arrow-down116cross-posted to: [email protected][email protected]
arrow-up1758arrow-down1external-linkElon Musk’s Grok Goes Haywire, Boasts About Billionaire’s Pee-Drinking Skills and ‘Blowjob Prowess’www.mediaite.comRandAlThor@lemmy.ca to Technology@lemmy.worldEnglish · 17 hours agomessage-square118fedilinkcross-posted to: [email protected][email protected]
minus-squarebrucethemoose@lemmy.worldlinkfedilinkEnglisharrow-up1·edit-24 hours agoYou should be running hybrid inference of GLM Air with a setup like that. Qwen 8B is kinda obsolete. I dunno what kind of speeds you absolutely need, but I bet you could get at least 12 tokens/s.
You should be running hybrid inference of GLM Air with a setup like that. Qwen 8B is kinda obsolete.
I dunno what kind of speeds you absolutely need, but I bet you could get at least 12 tokens/s.