Andy Reid@lemmy.world to Technology@lemmy.worldEnglish · 2 years agoAI companies are violating a basic social contract of the web and and ignoring robots.txtwww.theverge.comexternal-linkmessage-square166fedilinkarrow-up1939arrow-down19cross-posted to: [email protected][email protected][email protected][email protected]
arrow-up1930arrow-down1external-linkAI companies are violating a basic social contract of the web and and ignoring robots.txtwww.theverge.comAndy Reid@lemmy.world to Technology@lemmy.worldEnglish · 2 years agomessage-square166fedilinkcross-posted to: [email protected][email protected][email protected][email protected]
minus-squareLvxferre [he/him]@mander.xyzlinkfedilinkEnglisharrow-up1·edit-22 years agoYup, it’s the same approach as email spam traps. Except the naughty list, but… holy fuck a shareable bot IP list is an amazing addition, it would increase the damage to those web crawling businesses.
Yup, it’s the same approach as email spam traps. Except the naughty list, but… holy fuck a shareable bot IP list is an amazing addition, it would increase the damage to those web crawling businesses.