Piefed uses Python which is faster for development but the language is slower. Lemmy is built on Rust. I appreciate some features Piefed has but I do wonder about its scalability.
I thought Rust was faster for basically every metric?
The entire advantage of Python is supposed to be ease of development, in exchange for slower code execution. It is especially bad in terms of multiprocessing, which Rust is great at.
As I severely lack expertise on back-end development I asked for clarification to the forbidden oracle (AI) but it also told me that Rust is faster. I am not sure whether you feel like debunking an AI comment but if this is false I would love to hear why because from my current understanding Rust is always faster (for back-end development).
AI response
That statement is technically false, but it contains a grain of practical truth that often confuses people.
Here is the breakdown of why that statement is misleading and where the misunderstanding comes from.
The Verdict: False
Rust is almost invariably faster than Python, even in IO-bound tasks.
If you have a web server handling 10,000 concurrent connections that are all waiting on a database (pure IO-bound), the Rust server will use significantly less RAM and CPU to manage those “waiting” connections than the Python server.
Why the statement is wrong (The Technical Reality)
The argument assumes that “IO-bound” means the CPU does zero work. That isn’t true. Even in an IO-heavy application, the application server must do the following:
Event Loop Management
The server has to track which connections are waiting and which are ready to resume.
Rust’s async runtimes (like Tokio) are incredibly optimized and have near-zero runtime overhead.
Python’s asyncio loop has significant overhead because it is still running interpreted Python code to manage the tasks.
Serialization/Deserialization
When the database replies, the server receives raw bytes. It must turn those bytes into objects (JSON parsing, ORM model instantiation). This is CPU-bound work.
Rust does this near-instantly.
Python is notoriously slow at object instantiation.
The GIL (Global Interpreter Lock)
Even if your code is async, Python can only execute one bytecode instruction at a time per process.
If 100 requests come back from the DB at the exact same millisecond, Python has to process them one by one.
Rust can process them in parallel across all CPU cores.
Where the statement is “Practically” True (The Grain of Truth)
The person making that statement is likely conflating “faster” with “indistinguishable to a human.”
If a database query takes 100ms:
Rust overhead: 0.1ms → total response ≈ 100.1ms
Python overhead: 5.0ms → total response ≈ 105.0ms
To the human user, 100.1ms and 105ms feel exactly the same.
In this specific context, you could argue that Python is “just as fast” as Rust because the bottleneck is the database, not the language. But it is incorrect to say Python is faster.
The “Fediverse” Exception (Why it matters for PieFed)
The statement “web servers are IO-bound” is often true for something like a simple blog.
It is less true for the Fediverse.
ActivityPub (the protocol PieFed and Lemmy use) involves two things that are heavily CPU-bound, not IO-bound:
JSON Parsing
Fediverse servers throw massive JSON blobs at each other constantly.
Python is slow at parsing JSON compared to Rust.
Cryptography (RSA Signatures)
Every time a server sends a message to another server, it must cryptographically sign it (HTTP Signatures). Every time it receives a message, it must verify the signature.
Rust handles crypto operations natively and extremely fast.
Python relies on C-extensions (like cryptography), which are fast, but the overhead of calling back and forth between Python and C for every single request adds up.
Conclusion
The statement is false.
Rust is faster at raw execution.
Rust is faster at handling high concurrency (even IO-bound).
Python is only “faster” in one metric: development velocity – you can write the code faster in Python, but the code itself will not run faster than Rust.
No, python can be incredibly fast for IO when scaled properly.
You generally don’t run a single process or even program for serving websites. There are task queues, callbacks, microservices etc so the bottleneck is almost never the programming language itself but the tooling and Python’s tooling for web is still miles ahead. Thats why big project ship more Django than Rust and all AI training is running on Python not Rust etc.
Don’t get me wrong Rust is a brilliant language but Python can often be better.
Finally you can outsource high performance tasks to Rust or C from within Python rather easily these days.
Python is an interpreted language, which is fundamentally always slower than a compiled language like Rust. However the main performance bottleneck are actually sql queries, and I believe we make a lot more effort to optimize them compared to Piefed.
Not sure what you mean by “advanced caching”. There is some basic caching for data which rarely or never changes, for example most things in /api/v3/site. But other data like post listings change a lot, so caching is not an option and instead its a matter of optimizing the sql queries (using the right indexes, reducing the number of joins, making benchmarks and looking at query plans).
I thought the biggest problem for Python would be the GIL as it cannot share memory between processes and therefore needs to do use a database or other tool to share between them. Though in hindsight most web related services probably use databases to read and write data and this do not work out of shared process memory.
Threading from a single process is just a bad scaling strategy anyway so GIL is rarely an issue so you’re right most big web stuff does indeed use a database/queue/cache layer for orchestrating multiple processes.
The whole point is to not have a centralized system. Development is also part of that, so providing multiple front ends for the same community also works towards that goal.
Oh, I had made one for quokk.au a few months ago. It got lost during an update as I didn’t bother to save it, I can probably whip another one up this weekend if I remember.
Edit: This was a wip screenshot I took. It was never 100% the same, but yeah.
Well, for one, the old Reddit interface’s appeal is fairly limited in general. Most people use apps anymore, I feel like in general desktop users are not catered to even in default interfaces. Of people who do use the desktop, more of them are likely to be new(er) Reddit users who probably find the Lemmy/Piefed default UI to be superior to Reddit - but the pool of people who prefer old Reddit is dwindling, which doesn’t usually lead to continuing support of things.
As it stands, few Lemmy instances have it and in any case it feels pretty fragile. It has broken a couple times and I think been fixed, but this has led instances to drop it altogether (dbzer0 used to have it, for example, but the lead said it was too much of a hassle and its devs weren’t responding to his questions).
I’m fairly worried that 1.0 is gonna break it, tbh. It never gets any updates that I have noticed, and its functionality is a bit all over the place.
Yeah, sorry you’re right, like being able to get feeds for small communities, and that sort of thing. But all the actual content is the same, is what I meant. Piefed doesn’t provide any content that you can’t get on Lemmy, just a different way to view it (UI) and organize it (features).
It does have polls. Those are not visible if you use Lemmy, right?
It will also add support for viewing microblogging posts (a.k.a. Mastodon) sometime in the next year. Obviously, that’s still content you can see all of if you use both Mastodon and Lemmy, because Mastodon supports polls as well.
But PieFed is the only one that shows both everything Lemmy shows plus the polls.
Well, it works. But Piefed is developing more features quicker atm.
Piefed uses Python which is faster for development but the language is slower. Lemmy is built on Rust. I appreciate some features Piefed has but I do wonder about its scalability.
it really depends what’s the metric. Most web server stuff is IO-bound not compute-bound so Python can actually be faster than Rust.
I thought Rust was faster for basically every metric?
The entire advantage of Python is supposed to be ease of development, in exchange for slower code execution. It is especially bad in terms of multiprocessing, which Rust is great at.
As I severely lack expertise on back-end development I asked for clarification to the forbidden oracle (AI) but it also told me that Rust is faster. I am not sure whether you feel like debunking an AI comment but if this is false I would love to hear why because from my current understanding Rust is always faster (for back-end development).
AI response
That statement is technically false, but it contains a grain of practical truth that often confuses people.
Here is the breakdown of why that statement is misleading and where the misunderstanding comes from.
The Verdict: False
Rust is almost invariably faster than Python, even in IO-bound tasks.
If you have a web server handling 10,000 concurrent connections that are all waiting on a database (pure IO-bound), the Rust server will use significantly less RAM and CPU to manage those “waiting” connections than the Python server.
Why the statement is wrong (The Technical Reality)
The argument assumes that “IO-bound” means the CPU does zero work. That isn’t true. Even in an IO-heavy application, the application server must do the following:
Event Loop Management
The server has to track which connections are waiting and which are ready to resume.
asyncioloop has significant overhead because it is still running interpreted Python code to manage the tasks.Serialization/Deserialization
When the database replies, the server receives raw bytes. It must turn those bytes into objects (JSON parsing, ORM model instantiation). This is CPU-bound work.
The GIL (Global Interpreter Lock)
Even if your code is async, Python can only execute one bytecode instruction at a time per process.
Where the statement is “Practically” True (The Grain of Truth)
The person making that statement is likely conflating “faster” with “indistinguishable to a human.”
If a database query takes 100ms:
To the human user, 100.1ms and 105ms feel exactly the same.
In this specific context, you could argue that Python is “just as fast” as Rust because the bottleneck is the database, not the language. But it is incorrect to say Python is faster.
The “Fediverse” Exception (Why it matters for PieFed)
The statement “web servers are IO-bound” is often true for something like a simple blog.
It is less true for the Fediverse.
ActivityPub (the protocol PieFed and Lemmy use) involves two things that are heavily CPU-bound, not IO-bound:
JSON Parsing
Fediverse servers throw massive JSON blobs at each other constantly.
Cryptography (RSA Signatures)
Every time a server sends a message to another server, it must cryptographically sign it (HTTP Signatures). Every time it receives a message, it must verify the signature.
cryptography), which are fast, but the overhead of calling back and forth between Python and C for every single request adds up.Conclusion
The statement is false.
No, python can be incredibly fast for IO when scaled properly.
You generally don’t run a single process or even program for serving websites. There are task queues, callbacks, microservices etc so the bottleneck is almost never the programming language itself but the tooling and Python’s tooling for web is still miles ahead. Thats why big project ship more Django than Rust and all AI training is running on Python not Rust etc.
Don’t get me wrong Rust is a brilliant language but Python can often be better.
Finally you can outsource high performance tasks to Rust or C from within Python rather easily these days.
Python is an interpreted language, which is fundamentally always slower than a compiled language like Rust. However the main performance bottleneck are actually sql queries, and I believe we make a lot more effort to optimize them compared to Piefed.
That makes sense to me logically. Are there advanced caching techniques being deployed? I’m really curious about this.
Not sure what you mean by “advanced caching”. There is some basic caching for data which rarely or never changes, for example most things in
/api/v3/site. But other data like post listings change a lot, so caching is not an option and instead its a matter of optimizing the sql queries (using the right indexes, reducing the number of joins, making benchmarks and looking at query plans).Here is an issue on this topic: https://github.com/LemmyNet/lemmy/issues/5555
I thought the biggest problem for Python would be the GIL as it cannot share memory between processes and therefore needs to do use a database or other tool to share between them. Though in hindsight most web related services probably use databases to read and write data and this do not work out of shared process memory.
Threading from a single process is just a bad scaling strategy anyway so GIL is rarely an issue so you’re right most big web stuff does indeed use a database/queue/cache layer for orchestrating multiple processes.
So we should all just jump? Meh. Lemmy 1.0 is releasing soon which should help quite a bit.
AFAIK the two can federate with eachother so I’m glad we have both.
Thesis: Lemmy is better!
Antithesis: Piefed is better!
Synthesis: Bitch, they’re federated??
FUCK YOU HEGEL STOP FOLLOWING ME
Not necessarely that’s the beauty of the fediverse. You can enjoy PieFed growth and all it’s content but stick on Lemmy if you don’t feel like moving.
I vote use both
https://lemmy.ml/comment/22544926
Lemmy still has a bigger user base doesn’t it?
Well yes, but you can interact with lemmy on piefed.
If it’s true It’s the main reason to not even think to use piefed.
Not if you want to make use of some of Piefeds tools.
I have to check if Voyager iOS supports it
It does
I mean, I am reading this on PieFed and I do see your comment just fine. As you see mine.
Why is that a reason not to use PieFed? You can obviously just block all instances that run on Lemmy if you somehow don’t like their users!
Ok sorry, I just misunderstood. I thought you couldn’t read lemmy from piefed at all.
The whole point is to not have a centralized system. Development is also part of that, so providing multiple front ends for the same community also works towards that goal.
And boobs and wieners.
Doesn’t matter to me, I’m still using the old reddit theme on .world and that’s what I want to keep using.
Piefed or Lemmy, it’s the same content, so you’re arguing about either UI or philosophical differences (Lemmy devs are tankies or whatever).
I’d consider Piefed if they had an old reddit UI available, but that’s never gonna happen and I absolutely cannot stand either Lemmy’s or Piefed’s UI.
Oh, I had made one for quokk.au a few months ago. It got lost during an update as I didn’t bother to save it, I can probably whip another one up this weekend if I remember.
Edit: This was a wip screenshot I took. It was never 100% the same, but yeah.
Cool, well let me know if you do, I’ll join up. I’m sure if you do there would be several dozen people who would be thrilled! 😂
It looks great to be honest
Why would that never happen?
Well, for one, the old Reddit interface’s appeal is fairly limited in general. Most people use apps anymore, I feel like in general desktop users are not catered to even in default interfaces. Of people who do use the desktop, more of them are likely to be new(er) Reddit users who probably find the Lemmy/Piefed default UI to be superior to Reddit - but the pool of people who prefer old Reddit is dwindling, which doesn’t usually lead to continuing support of things.
As it stands, few Lemmy instances have it and in any case it feels pretty fragile. It has broken a couple times and I think been fixed, but this has led instances to drop it altogether (dbzer0 used to have it, for example, but the lead said it was too much of a hassle and its devs weren’t responding to his questions).
I’m fairly worried that 1.0 is gonna break it, tbh. It never gets any updates that I have noticed, and its functionality is a bit all over the place.
Well no, it’s not just UI. Piefed has more features than Lemmy right now.
Fair enough if you’re tethered to lemmy due to an old-reddit style UI, but it’s not purely rooted in UI and ideology.
Yeah, sorry you’re right, like being able to get feeds for small communities, and that sort of thing. But all the actual content is the same, is what I meant. Piefed doesn’t provide any content that you can’t get on Lemmy, just a different way to view it (UI) and organize it (features).
It does have polls. Those are not visible if you use Lemmy, right?
It will also add support for viewing microblogging posts (a.k.a. Mastodon) sometime in the next year. Obviously, that’s still content you can see all of if you use both Mastodon and Lemmy, because Mastodon supports polls as well.
But PieFed is the only one that shows both everything Lemmy shows plus the polls.