And see if they’re making repeated un-scoped (no page paramater) requests to /api/v3/comment/list. If they are, block them in your firewall.
Those used to hit my instance constantly with requests like /api/v3/comment/list?sort=Old&page=16514 (yes, page 16,414). When I blocked those IPs making those requests, problem solved.
I don’t see those specific IPs, nor 16514. But now I see what scrapers tend to look like in the logs :)
I am now pretty sure that the cause was scraper-like activity coming from the Mlmym front-end that I am serving over an onion site. I am not sure if it randomly started mis-behaving or if a tor scraper was using it.
After blocking this, federation was restored, performance increased, and CPU use came down:
Check your web access logs for these 3 IPs:
And see if they’re making repeated un-scoped (no
pageparamater) requests to/api/v3/comment/list. If they are, block them in your firewall.Those used to hit my instance constantly with requests like
/api/v3/comment/list?sort=Old&page=16514(yes, page 16,414). When I blocked those IPs making those requests, problem solved.Thanks!
I don’t see those specific IPs, nor 16514. But now I see what scrapers tend to look like in the logs :)
I am now pretty sure that the cause was scraper-like activity coming from the Mlmym front-end that I am serving over an onion site. I am not sure if it randomly started mis-behaving or if a tor scraper was using it.
After blocking this, federation was restored, performance increased, and CPU use came down: