Andy Reid@lemmy.world to Technology@lemmy.worldEnglish · 1 year agoAI companies are violating a basic social contract of the web and and ignoring robots.txtwww.theverge.comexternal-linkmessage-square168fedilinkarrow-up1939arrow-down19cross-posted to: [email protected][email protected][email protected][email protected]
arrow-up1930arrow-down1external-linkAI companies are violating a basic social contract of the web and and ignoring robots.txtwww.theverge.comAndy Reid@lemmy.world to Technology@lemmy.worldEnglish · 1 year agomessage-square168fedilinkcross-posted to: [email protected][email protected][email protected][email protected]
minus-squareLvxferre [he/him]@mander.xyzlinkfedilinkEnglisharrow-up1·edit-21 year agoYup, it’s the same approach as email spam traps. Except the naughty list, but… holy fuck a shareable bot IP list is an amazing addition, it would increase the damage to those web crawling businesses.
Yup, it’s the same approach as email spam traps. Except the naughty list, but… holy fuck a shareable bot IP list is an amazing addition, it would increase the damage to those web crawling businesses.