Hello everyone!
It’s time for another long-overdue update on how Fedecan and our various sites are doing. It’s been just over two years since the great Reddit migration, and in that time we’ve made some solid progress:
- Registered Fedecan.ca as a non-profit
- Moved from OVH to our own hardware in a Vancouver datacenter
- Started collaborating with the team behind sh.itjust.works
- Launched Pixelfed.ca
- Launched Piefed.ca
Finances
Here’s a look at our bank balance since we began accepting donations:
We’re currently sitting at around $2,900, with a monthly burn of about $200, which gives us roughly a year of runway. We have some additional annual costs (like domain renewals and non-profit registration), but overall we run very lean.
Fedecan still owes:
- TruckBC: $1,980
- Shadow (me): $525
These were out-of-pocket hosting and non-profit registration costs from 2023/2024. It’d be great to get those covered, but we want to keep at least a year of operating expenses in reserve.
If you’re a regular user and value what we’re doing, please consider donating! We have multiple ways to donate, you can find the comparison and donation links on our website: https://fedecan.ca/en/donate
Sh.itjust.works
Nothing major to report here - we’ve all been a bit busy lately, but collaboration is continuing slowly behind the scenes.
Fediverse Growth
We’re seeing a healthy volume of posts and communities on lemmy.ca, surging with each Reddit drama:
Infrastructure
Our server is a Dell R7515 with an EPYC 7763, 1 TB ram and 4x 7.68tb nvme data disks, which is hosted in a datacenter in Vancouver, BC.
I spun up victoriametrics + victorialogs a few weeks ago and have been ingesting all of our data, giving us the ability to put together some nice grafana dashboards.
Everything is running great on the infrastructure side of things. Our server is barely working up a sweat and we shouldn’t have to worry about scaling for a long time.
Lemmy.ca still comprises almost all of our traffic:
Lemmy.ca
Our over provisioned stack is performing well, handling the occasional lemmy / lemmy-ui dropout:
Similarly the DB is mostly running out of ram:
Our object storage is slowly climbing as expected, but we’ve got several years of capacity to figure out a long term solution:
I’m also doing some limited analytics on our web logs. As expected, lemmy.world makes up the majority of our federation traffic:
One interesting thing to see from the user-agent data is the breakdown of traffic by the different mobile clients:
The “dart” UA is just a common web library, Thunder reports as this and I suspect other clients do too. If you’re a client developer, please set your user-agent!
Out of the alternative web clients we support, tesseract is the most popular although the overall traffic volume is still low:
We only store 7 days of logs but I’m hoping to get these pulled out into metrics soon, since it would be interesting to track which clients / interfaces people use over time.
Pixelfed.ca
Not much to say on this one, due to using local storage it currently runs on a single VM without redundancy.
Piefed.ca
Piefed runs on a pair of VMs with its own database and object storage backends.
Cloudflare
If you want to compare against previous data posts, here’s our same cloudflare graphs for lemmy.ca
As always, feel free to reach out if you have any questions or ideas. Thanks for being a part of the Fediverse!
As requested, here’s some additional info about donation methods / sizes / volume!
PS: are the recurring ones per month or per year?
Monthly. There were a couple yearly that I just normalized down to monthly.
Thank you so much! If I understand correctly it’s around 80 donors, counting recurring and one-time together?
ish
Thanks, that’s quite informative. I take it the Stripe option is new. If those trends continue, it looks like things are currently financially secure, but not necessarily paying things back soon or building much of a reserve for replacement hardware. Hopefully the added awareness gets a few more donors. Given the low costs we’re seeing, even $10/year will make a notable difference.
No we launched with stripe, but if you look at our site you’ll see we don’t push it heavily since we pay more fees there.
Request for info to be added to fundraising page: which fundraising payment options are processed by American companies. If they’re taking fees, I’d like those fees to go to Canadian firms.
Fuck it, sending an e-transfer. ;)
E: sent $100. The equivalent of two one-year reddit subscriptions. I cancelled my subscription to move over here. Good luck with the fundraising :)
There’s a reddit subscription? Whatever for??
Before the enshittification took root, reddit was where I interacted the most with people online. You could buy a subscription to get an ad-free experience. It came with a bunch of Reddit gold you could hand out. Basically, help pay for server time, and get an ad-free experience without adblocking.
The amount of time I spent on Reddit I would compare to a Netflix subscription or similar.
In the ramp up to their IPO when they started to fuck with everything, my goodwill evaporated. I stopped moderating and abandoned my subs. Whenever I had new content I wanted to post, I instead post to the fediverse. So, there ya go 😅
Thanks for taking the time to answer!
And wow, you mod a bunch of communities here. I just joined a few of them that piqued my interest! I’ve been meaning to learn how to play DnD with my kids.
I actually locked the D&D community on lemmy.ca that I created once the ttrpg.network D&D community took off. I recommend that one. I could reactive the lemmy.ca if someone went sideways with that instance.
I also recommend the lemmyverse.net website for finding communities to subscribe to. Copy and paste the community name – should look something like [email protected] – into the search box and then subscribe.
Also, try to subscribe to communities that aren’t on lemmy.world, even if one exists on lemmy.world. Let’s avoid putting all of our eggs in one basket ;)
Thanks again for the great advice!
Good idea, they all do except interac since everything routes through stripe. It’s absurd the amount of transaction data they get. Thanks!
Thank you for keeping this instance alive, Also nice transparency very appreciated
Appreciate you sharing the data like this. It’s cool to peek behind the curtain!
Is there any regular offsite backup of data?
Yes, shipped to a server at my house.
I’m a sysadmin by day and… Only one host? And it needs 1TB of RAM? Seems excessive. Probably not though. IDK how optimized Lemmy is.
There’s a lot I could say here but I’ll limit myself to this: the proof is in the pudding. Lemmy.ca is working well. You all should be very proud of what you’ve done.
Two hosts would cost more in colo fees and we don’t need the resources. The redundancy would be nice, but this is enterprise grade hardware covered under Dell prosupport. I also have an identical box running my home systems, so I could steal parts temporarily if we had an issue.
Yeah the ram is just what the hosts came with. It’s way more than we need and so everything is very over provisioned.
All fair. I’m not trying to say you’re doing it wrong at all, quite the opposite.
And yes, redundancy is nice, but it really depends on the importance of the data on the system and the budget.
To be blunt: if Lemmy.ca goes down for any length of time, that would suck for everyone here, but nobody will die, there won’t be any loss of profits or whatever… In business talk, the risk of what could be lost due to an outage is less than the cost of the hardware to prevent an outage.
I understand your position and an in warranty Dell server system isn’t cheap.
What you’re currently going is clearly working. So I don’t have any complaints.
It doesn’t need it. That hardware is just what was available.
That’s really cool that admins can tell which clients folks are using, that would be very helpful for developers to gauge user satisfaction.
Sent you something in addition to the recurring to help with that repaying. ☺️
Thank you! I got here at the great Reddit migration, all this time lemmy.ca has been extremely reliable. I’ve had a great time. Just want to say I appreciate the time, knowledge and effort you put in it.
Thank you for the great job! 🙏 🚀
Let me ask two explicit questions:
-
Considering costs and total number of users, how much should be a user’s monthly donation to keep things even or a little on the safe side?
-
Considering costs and total number of donators, how much should be a donator’s monthly donation to keep things even or a little on the safe side? This is a more realistic estimate, as there are users (say, students) who can’t pay (and of course users who simply don’t want to pay).
Many Fediverse initiatives seem too shy to give this kind of information, but I think there’s nothing wrong about it. Please tell us in time if the economy were to be going bad, nobody wants another lemm.ee event :) As Impossible Mission for the Commodore 64 used to say:
Stay awhile, stay forever!
I have no problem sharing anonymized info for things like this, it’s just difficult to pull.
We accept donations a few different ways, they can be one time or recurring, we don’t know who people are , and we often get paid weeks after they happen once it gets reconciled by stripe.
Let me see what I can do.
Re: Lem.ee that’s kinda a different problem. They weren’t financial issues afaik, but admin burnout. I don’t think that’s a concern on our side at this time.
We can get a rough estimate for your first question with the information we have. They’ve shared that it costs them about $200/month, and we can see from the sidebar that we have 3k users per 6 months (estimate for number of active users). That means approximately 7c/month per user.
Cheers! Then it’d be quite cheap if every user gave their 10c/month. Let’s see what they say about actual donor-users.
I’d also like this info. From what we can see, donations are pretty stable relative to expenses, which is another way of saying those personal expenses are never getting repaid. @[email protected] can we set up a repayment fundraiser so active users can be incentivized to at least pay that down? I’d also like to know how many active donors there are, which I also know is difficult with more-or-less annual donors like myself requiring at least 12 months of donations to be reviewed, but it would give us a better appreciation of the monthly or annual donations per donor required to keep this afloat.
And, as always, thank you for the work you and the rest of the administrators do to provide us with this space.
I like the idea of a specific fundraiser.
Our post 6 months ago is what drive the big spike visible on the graphs, so I was hoping for a similar boost this time.If we don’t get there naturally, I think that’s a reasonable next step.See below for your other question.
Edit: I was wrong, the big spike was Smorks transferring our old donations over. I forgot about that!
Yeah, I didn’t donate for that last post, but this one was a good reminder to do the annual donation again. I’ll be watching for a post about that fundraiser, too. Hopefully it isn’t needed, but I will definitely put in extra to pay you all back.
-
I tried piefed.ca but it was pretty broken. Importing filter blocks will cause internal server errors and fail every time. Some communities including the emulation community I moderate on sh.itjust.works show up blank except stickies. Constant random connection errors similar to lemmynfsw.
Was this recently? I just pulled in some more fixes a few hours ago. It’s still under very active development.
Hello Shadow! Just chiming in that import settings is also broken for me. The json file is about 35 kb or so. It just tells me ‘this might take a few minutes’ but nothing imports. I tried a few times yesterday and again just now. It works for my Piefed.social account though.
As always, thanks for all you do. :)
Willing to email me your file so I can try? [email protected]
Turns out I’m suddenly not busy now. Saw your reply over there but I’ll just follow up here.
Importing works now!
My community is still blank. I checked to make sure I didn’t fat thumb selecting a language in my profile settings or a block but I don’t see anything weird. :s
Edit: oh right another thing I noticed was if my piefed.ca account replies to a sticky there it’s not federated, or I can’t see it on my other accounts anyways.
No a couple weeks back now. I made this account last month before I realized .ca existed. None of these issues were present on .social. That’s more accurate on the timeline now, edited for bad memory.
I might return to .ca later once it’s more mature, kept the account. I’ll check to see if anything changed later after work :)
Thank you for the update!
