Hello World!
We’ve recently added PieFed.World to the Fedihosting Foundation portfolio.
PieFed.World is still in its early stages, and we still need to port some of our automations we already have in place on Lemmy.World. This includes functionality to inform people about moderation actions taken against them, as well as some other moderation tooling. Administration is currently done by the same team responsible for Lemmy.World, and the same rules that apply to Lemmy.World also apply to PieFed.World.
What is PieFed?
PieFed is a Fediverse/Threadiverse platform similar to Lemmy or Mbin/kbin. You can find a description and feature comparison with Lemmy on their website.
While PieFed has a range of features currently not present in Lemmy, it also is a a lot younger and isn’t quite as robust as Lemmy currently is. There are still many bugs and missing features that you will likely run across compared to Lemmy, which will take time to be addressed. PieFed has fairly active development and is seeing a lot of issues addressed fairly quickly, which is especially important recently, as the number of active PieFed instances and PieFed users increased significantly with a range of Lemmy instances opening up PieFed instances as well. PieFed currently does not have proper “stable” releases and no test suite, so it’s not unlikely for things to break from time to time. Although 1.0.0 has already been released a while back, there are still too many issues addressed in more recent commits to stay on that version.
As PieFed is part of the same federated network as Lemmy and Mbin, all PieFed communities can be accessed from Lemmy and Mbin, as well as other Fediverse platforms. Likewise, PieFed can access communities from Lemmy, Mbin and other Fediverse platforms. Whether you use a PieFed instance, a Lemmy instance, or an Mbin instance, it does not matter what type of instance the community is on. The software affects your own user experience, but the content is available regardless.
Creation of communities
Creation of communities will be limited to admins for the first week of the public launch. We will reserve this time to allow community moderators of established communities to claim the name on PieFed.World before we open community creation to the public. We will limit this to communities with the same name and at least 2k monthly active users. In case of multiple qualifying communities with the same name on different instances expressing interest, Lemmy.World communities will be given preference, afterwards the number of monthly active users. Please reach out if you’d like to discuss an exception. Requests can be posted in [email protected]. After the first week, community creation will be available to anyone.
Migration of communities
PieFed has a feature to migrate communities to a local instance. We will not be offering PieFed’s community migration feature initially.
We still need to research the details of how this works and the impacts this has on federation before we will make a decision on whether will support this in the future. If requested, we may reserve some names for potential future community migrations until we have made a decision to allow community migrations.
This does not prevent you from moving communities in the classic way, by opening up a new community and posting in the old community that people should move over.
Private voting
We had previously disabled private voting for PieFed.World before opening the instance to the public, as the original implementation has a range of drawbacks when it comes to federation, and our team overwhelmingly believed that the individual benefits of private voting did not outweigh the impact this has on the Fediverse beyond the user’s instance. Additionally, due to the implementation of that feature, it was also trivial to identify the original voter, which significantly limited the promises of this bringing actual voting privacy.
Since then, the implementation of private voting has been changed to provide the option of federating or not federating votes. While this is more likely to result in vote differences across instances, it does not feed bad information to other instances, which could make it a lot harder for other instances to identify manipulation.
Non-federated voting is available for all PieFed.World users.
Topics
Topics are a kind of “starter packs” or collections grouping multiple communities that people can follow, curated by the admin team. We don’t have a clear vision for the structure of these yet.
You can see an example structure on piefed.social.
Feel free to let us know your thoughts on this.
Feeds
PieFed supports feeds, which are user-created groups of communities, similar to topics. These are currently in a global namespace and all users can create public feeds in the same shared namespace.
Reputation and vote weight
PieFed has options for admins to treat certain types of content differently for “reputation” calculation, as well as options for weighing votes of specific instances differently compared to others. We currently have all options for treating certain content, communities or instances differently disabled.
How does PieFed compare to Lemmy?
PieFed has various features not present in Lemmy, check out their website!
There is also various functionality that Lemmy has, which you may be missing currently with PieFed for now:
Limited API support
In Lemmy, the default web interface relies entirely on the Lemmy API. This has the major benefit of all functionality available in the default web interface also being available to all third party clients. PieFed currently uses separate code paths and implementations for the default web interface and its API. To make it possible to access functionality in third party apps, dedicated API endpoints have to be created, even if this functionality is already available in the default web interface. This also includes alternative web-based UIs.
Multiple developers of alternative UIs and mobile clients are already working on PieFed support, some already released experimental versions.
Limited availability of Markdown previews
Markdown previews are currently only available in posts. There are many other places that accept markdown, but you can’t preview the rendered comment before submitting it. This is tracked in #532.
Image uploads only on post creation
Images can’t be uploaded to comments currently. You’ll have to host them externally for now. This is tracked in #768.
Autocompletion of users/communities
Usernames and communities can’t be autocompleted when typing their names currently. This is tracked in #799.
Limited availability of modlog
Modlog is currently very limited. While there is an instance modlog, there are currently no filters available, so it’s not possible for users to see actions taken against a specific user or within a specific community. Community modlog exists, but it is currently only available to community moderators and admins. Filtering modlog is tracked in #846.
Moderator hierarchy
Lemmy has a moderator hierarchy based on the time a moderator was appointed, relative to other moderators in the community. This allows moderators to add other moderators, but they can only remove moderators that were added later than they were. There are a few other actions that check moderator hierarchy as well, including deletion only being possible by the top mod. In PieFed, communities have one or more owners, who can add and remove moderators, while all other moderators are currently on equal level. Community owners currently cannot be changed without editing this directly in the database, if you’d like to change owners in your community please reach out in [email protected].
Donations
Similar to Lemmy, PieFed development is supported by donations. You can donate to PieFed development through Patreon.
Additionally, we would appreciate donations towards the Fedihosting Foundation, the non-profit organization operating PieFed.World, Lemmy.World, and a range of other Fediverse platforms.
Problems and questions
Please report any issues and questions about PieFed.World in [email protected].
For topics about the software PieFed, please visit [email protected].
Bugs can be reported on Codeberg.
TLDR: New platform with similar functionality available, Lemmy.World will continue to exist.
edit: reordered sections and minor wording changes
edit 2: updated community owner information
You must log in or register to comment.
I’m grateful for all the work y’all do to keep making the fediverse cooler.
The way they implement Feeds and Topics is very interesting. They manage to get rid of all the different fediverse servers having their own community and mash them all in one combined feed or topic. I would love to see this “feed” option or similar being implemented on Lemmy to abstract away all the defederation layers.
Looking at the rest of the featureset of Piefed it looks very much inferior to Lemmy. While their choice of Python instead of Rust would theoretically provide them faster time to market, Lemmy which is on Rust already exists and is functional. Piefed is half functional and has worse scaling potential.
Most other differences appear to be more how certain features are implemented such as
Lemmy has a moderator hierarchy based on the time a moderator was appointed, relative to other moderators in the community. This allows moderators to add other moderators, but they can only remove moderators that were added later than they were
and
Improved hotness ranking algorithm (subjective)
Which really begs the question why a whole new platform is needed instead of simply implementing these features on Lemmy.
Which really begs the question why a whole new platform is needed instead of simply implementing these features on Lemmy.
Technical reasons aside, there’s value in having a variety of platforms, so not all features are gated behind the decisions of the Lemmy maintainers.
- Consolidated comments views for all crossposts
- Actual instance blocking
- Keyword filters
- Built-in post schedulers
- Community flairs
All of those features exist on PieFed and not Lemmy, so I wouldn’t say that Piefed is that behind features wise.
There are still improvements for sure, but there are also areas where Piefed is quite ahead
Written in a common programming language that many developers understand and which has a bright future ahead of it. Python, of course! This will enable more contributions from a wider range of people than if it was made with Erlang, Ruby, Rust or PHP, for example.
Consume few resources, to make it cheap to run. Many examples of federated software are bloated Rube Goldberg machines that require hefty servers and serious server administration skills, making money a constant problem. PieFed instances will be small and nimble.
Seems orthogonal?
I am 100% for the accessibility of Python, but won’t scaling up to large instances/userbases be an issue? Unless some underlying library does the heavy lifting I guess, or they eventually use some optimized subset. Or maybe the bulk of the costs is disk/bandwidth anyway, not CPU, IDK.
These features though:
Communities are organized into topics. See https://piefed.social/topics.
Image-heavy communities can have a tiled/masonry view, like https://piefed.social/c/[email protected]
People who get downvoted a lot end up with a ‘low reputation’ indicator next to their name. You’ll know it when you see it.
Hide all posts based on keyword filters.
Upvotes in meme communities do not add to reputation.
Improved hotness ranking algorithm (subjective)
Voting is private.
All these are fire! I have no idea why Lemmy devs wouldn’t implement them (and am open to explanations/arguments).
And this is good too, though it makes me worry about power-trippin’ admins: https://join.piefed.social/2024/06/22/piefed-features-for-growing-healthy-communities/
Python, of course! This will enable more contributions from a wider range of people than if it was made with Erlang, Ruby, Rust or PHP, for example.
Rust is probably the most high performance future-proof choice language of choice. While still not extremely popular, it is steadily growing. Of course the way code is written is more important than the language. But the fact that Lemmy is already on Rust and Piefed on Python, puts Piefed a foot behind unless their code uses far more efficient algorithms.
Voting is private.
This is an awful idea because bots become invincible. Also how would this even be possible in the fediverse? Publishing votes is required for federation. Even if it is hidden publicly, someone could make an instance which federates with all other instances and simply see the votes in their back-end.
People who get downvoted a lot end up with a ‘low reputation’ indicator next to their name. You’ll know it when you see it.
This promotes hiveminding.
Improved hotness ranking algorithm (subjective)
Probably good
The best thing in their list seems to be community migration of accounts but they have not implemented that yet.
Python, of course! This will enable more contributions from a wider range of people than if it was made with Erlang, Ruby, Rust or PHP, for example.
private voting
After having discussed with a few people, Rimu agreed to drop the private voting. The system will switch to two votes button
-
a local one that will stay local
-
a federated one that will be sent to other instances
-
https://chat.piefed.social/#narrow/channel/5-admins/topic/Private.20voting/near/332
This provides better transparency than Lemmy, as users are now complaining about tool like lemvotes:
This provides better transparency than Lemmy, as users are now complaining about tool like lemvotes: https://lemmy.dbzer0.com/post/48233901/19876557
People are complaining because… they have been detected for vote botting? I think this kind of proves my point. With public votes the users can see whether any manipulation is taking place. Because the admins do not seem to take it into account.
Two sets of vote buttons sounds obtuse. If users really do not want their votes published a simpler option would be to have a toggle in their profile or on the header bar to decide whether their votes will be local or global
The issue comes more from users not being aware that federated votes are public due to federation.
There is indeed a checkbox in the settings
https://chat.piefed.social/#narrow/stream/3-general/topic/Public.20voting.20UI/near/2125
Votes UI
https://chat.piefed.social/#narrow/stream/3-general/topic/Public.20voting.20UI/near/2132
I don’t think “people are complaining” is correct. Just one person started complaining because they were caught using that tool and they were caught because they were harassing users and their harassments were getting a weird amount of upvotes for being a piece of shit. A couple other people half-ass agreed with them, but there’s never been any major complaints or movements.
Saying “people are complaining” because of one unhinged person is highly inaccurate.
https://lemmy.dbzer0.com/post/43096615?scrollToComments=true
Lemmy.ml has defederated gregtech.eu , the instance used for lemvotes
-
when we started setting up piefed.world we subscribed to all active communities known to LW, which allowed us to already run with production level load on the inbound federation side. we also already discovered various performance bottlenecks that have since been improved, but i expect there to be plenty more to get addressed.
in its current state, i don’t think piefed actually performs better than lemmy when exposed to the same traffic, but i don’t have any numbers to confirm this. a large part of performance optimization can be done even without switching languages though, simply by structuring your code correctly.
when really optimizing both languages for performance, rust would easily outperform python, but it depends a lot on the workload whether this is relevant. ensuring that the database structure and queries are optimized is just as important as the application itself.
in its current state, i don’t think piefed actually performs better than lemmy when exposed to the same traffic, but i don’t have any numbers to confirm this.
we also already discovered various performance bottlenecks that have since been improved, but i expect there to be plenty more to get addressed.
It’s interesting that it’s even in the ballpark at this early stage! And it’s great that they get a ‘real world’ stress test to see what paths chug.
but it depends a lot on the workload whether this is relevant.
Yeah, this is what I was thinking!
If piefed ends up having a few CPU hot spots, they could theoretically ‘break out’ of Python just for them and/or get replaced with select precompiled libraries. Python is great orchestration language, hence that’s precisely what many projects do.
There’s also the occasional effort at a sped-up python runtime, like Pyston, Mojo, GraalVM and such. It’s possible that could be an option for piefed some day.
And like you said, database structure is more important, and runtime overhead may not be too relevant.
choice of programming language for web apps often has minor effects on scalability if any, it’s all about the design and database
and yea these instances need a lot of disk space, but programming language doesn’t affect that either
private voting
After having discussed with a few people, Rimu agreed to drop the private voting. The system will switch to two votes button
-
a local one that will stay local
-
a federated one that will be sent to other instances
-
https://chat.piefed.social/#narrow/channel/5-admins/topic/Private.20voting/near/332
This provides better transparency than Lemmy, as users are now complaining about tool like lemvotes: https://lemmy.dbzer0.com/post/48233901/19876557
-
From my experience contributing to Piefed, the bottleneck is almost always the database. So, doing optimization on that front has yielded improvements. I suspect that the db is likely the main bottleneck of lemmy as well (though I know network speed/latency has been an issue in the past).
NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O lemmy-postgres-1 1717.01% 48.6GiB / 78.12GiB 62.21% 1.84TB / 26.9TB 59.2TB / 67.7TBthe container has been up for 3 weeks
I was referring more to responsiveness and page loading time than overall resource usage. I don’t know enough on the sysadmin or database side of things to really speak authoritatively about things at scale. I just mean that it’s usually postgres instead of the language of python that limits what I have worked on.
i guess to put it in perspective i should have included other containers as well.
this is on a 32 physical core box with 96gb of memory:
NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O lemmy-lemmy_ui_1-1 13.59% 1.864GiB / 94.15GiB 1.98% 198GB / 1.34TB 5.5GB / 4.45GB lemmy-lemmy_ui_3-1 20.12% 2.764GiB / 94.15GiB 2.94% 312GB / 2.22TB 6.36GB / 4.24GB lemmy-lemmy_ui_2-1 54.07% 2.042GiB / 94.15GiB 2.17% 222GB / 1.52TB 5.61GB / 4.61GB lemmy-lemmy_api_3-1 10.16% 3.629GiB / 94.15GiB 3.85% 5.6TB / 1.71TB 290GB / 411GB lemmy-lemmy_sched_1-1 0.01% 42.79MiB / 94.15GiB 0.04% 9.72GB / 1.23GB 1.38GB / 2.56GB lemmy-lemmy_api_2-1 30.97% 4.049GiB / 94.15GiB 4.30% 5.07TB / 1.57TB 281GB / 401GB lemmy-lemmy_api_1-1 27.39% 3.913GiB / 94.15GiB 4.16% 4.95TB / 1.54TB 286GB / 404GB lemmy-postgres-1 1500.05% 49.99GiB / 78.12GiB 63.98% 1.84TB / 27TB 59.2TB / 67.7TB lemmy-lemmy_fed_1-1 405.39% 802.1MiB / 94.15GiB 0.83% 3.67TB / 5.67TB 12.9GB / 19.4GB lemmy-pictrs-1 514.29% 1.495GiB / 94.15GiB 1.59% 8.79TB / 5.36TB 511GB / 114TB lemmy-pictrs_db-1 2.76% 2.956GiB / 68.36GiB 4.32% 531GB / 325GB 13.1TB / 9.25TBwith lemmy it’s a lot easier to separate frontend performance from backend performance, with piefed that’s harder to tell when not just using the APIs and requires more analysis. proper indexes and well crafted queries have a lot of impact though. most logic that you can move to the db side is probably better done there, but it can make queries a lot more complex.
Oh that’s fascinating. I wonder what the hot code for the postgres instance would be like in a profiler, or if it would ‘like’ a SMT-heavy server like a IBM Power10 one since most of the load is db calls.
…Also, what is lemmy-pictrs doing? Is it just serving, or does it transcode/optimize too?
pict-rs also does transcoding and resizing of thumbnails, yeah. it caches the results in s3, but it there is lots of new content all the time obviously. i did pick one of the more active load snapshots, it’s not always at 500%.
Random thing, but if you are looking for lossless png/jpeg optimizers, I did a big shootout a while back and found this little nugget:
https://github.com/fhanau/Efficient-Compression-Tool
It often beats the filesizes of big (zopfli) PNG compressors with a fraction of the CPU time. It can also losslessly convert jpegs to a progressive format, so they load better on the user’s end (and also shave off like 1% of the filesize).
Or maybe you already handle this! IDK, I am not an admin :)
Piefed’s UI is noticeable slower/laggier than lemmy.
Yes it’s horrendous. Taking me 20-30 seconds to load a page whereas on Lemmy it’s instant.
I’m on piefed.social. What differences are there, if any?
we haven’t really rolled out topics yet, .social already has many communities sorted into topics.
the biggest difference, also longer term, is probably the way reputation is calculated. for example, piefed.social has all these checkboxes checked as far as i know:
we’re also working on getting our lemmy automod compatible with piefed to ensure that e.g. spam is removed on both instances when seen.
i don’t know enough about piefed.social instance level moderation, but there are likely some differences there as well.
Whoops, wish I knew some of this stuff before. Thanks for description.
We will reserve this time to allow community moderators of established communities to claim the name on PieFed.World before we open community creation to the public. We will limit this to communities with the same name and at least 2k monthly active users. In case of multiple qualifying communities with the same name on different instances expressing interest, Lemmy.World communities will be given preference, afterwards the number of monthly active users.
I don’t quite understand the point of this. Are you trying to encourage migration? Running two of the same community with the same mods? Doesn’t make sense to do to me in a different domain. Not like users want to subscribe to two of the exact same community down to the mods.
we want to provide the option of migration for communities interested in that without having someone just create a ton of communities with popular names just to grab the names.
Why is posting from a VPN blocked in lemmy.world? What the hell guys?
I have literally never not posted from a VPN…
I’m always on a VPN these days and it’s obnoxious to have to turn it off to shit post :(
see here: https://lemmy.world/post/11967676
I think that votes should always be public, but if you want privacy, then they should be anonymized. This is how PieFed works, but with votes being hidden as well. If they are anonymized already, then what’s the benefit to keeping them private? It’s far too common for admins and mods to just not take notice of vote manipulation, and that’s very understandable. However, this can be helped by letting anyone who has a suspicion take a brief look and verify. Reporting based on a suspicion is just not a good solution, because it will create too many false reports and prevent people from even bothering to report just a suspicion.
It’s just sad how common it is, and it can have some rather frustrating consequences.
This is awesome, much love!
Nice !
