Lemmy
  • Communities
  • Create Post
  • Create Community
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
fattyfoods@feddit.nl to Open Source@lemmy.ml · 22 小时前

The Open-Source Software Saving the Internet From AI Bot Scrapers

www.404media.co

external-link
message-square
83
fedilink
  • cross-posted to:
  • [email protected]
  • [email protected]
462
external-link

The Open-Source Software Saving the Internet From AI Bot Scrapers

www.404media.co

fattyfoods@feddit.nl to Open Source@lemmy.ml · 22 小时前
message-square
83
fedilink
  • cross-posted to:
  • [email protected]
  • [email protected]
Anubis, which block AI scrapers from scraping websites to death, has been downloaded almost 200,000 times.
  • Captain Beyond@linkage.ds8.zone
    link
    fedilink
    arrow-up
    25
    ·
    18 小时前

    Yes.

    Source: I use it on my instance and federation works fine

    • bdonvr@thelemmy.club
      link
      fedilink
      arrow-up
      13
      ·
      18 小时前

      Thanks. Anything special configuring it?

      • Captain Beyond@linkage.ds8.zone
        link
        fedilink
        arrow-up
        13
        ·
        edit-2
        17 小时前

        I keep my server config in a public git repo, but I don’t think you have to do anything really special to make it work with lemmy. Since I use Traefik I followed the guide for setting up Anubis with Traefik.

        I don’t expect to run into issues as Anubis specifically looks for user-agent strings that appear like human users (i.e. they contain the word “Mozilla” as most graphical web browsers do) any request clearly coming from a bot that identifies itself is left alone, and lemmy identifies itself as “Lemmy/{version} +{hostname}” in requests.

Open Source@lemmy.ml

opensource@lemmy.ml

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: [email protected]

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

  • Open Source Initiative
  • Free Software Foundation
  • Electronic Frontier Foundation
  • Software Freedom Conservancy
  • It’s FOSS
  • Android FOSS Apps Megathread

Rules

  • Posts must be relevant to the open source ideology
  • No NSFW content
  • No hate speech, bigotry, etc

Related Communities

  • [email protected]
  • [email protected]
  • [email protected]
  • [email protected]
  • [email protected]

Community icon from opensource.org, but we are not affiliated with them.

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 792 users / day
  • 1.48K users / week
  • 2.91K users / month
  • 10.5K users / 6 months
  • 1 local subscriber
  • 38.7K subscribers
  • 2.34K Posts
  • 34.2K Comments
  • Modlog
  • mods:
  • Evan@lemmy.ml
  • kevincox@lemmy.ml
  • CrypticCoffee@lemmy.ml
  • Lettuce eat lettuce@lemmy.ml
  • BE: 0.19.9
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org