It’s not, by far. But vllm or SGLang don’t support switching the model… such a shame.
- 0 Posts
- 2 Comments
Joined 9 days ago
Cake day: January 28th, 2025
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
Vllm unfortunately doesn’t support switching the model without a restart.