• 0 Posts
  • 76 Comments
Joined 1 year ago
cake
Cake day: July 22nd, 2023

help-circle









  • robots.txt does not work. I don’t think it ever has - it’s an honour system with no penalty for ignoring it.

    I have a few low traffic sites hosted at home, and when a crawler takes an interest they can totally flood my connection. I’m using cloudflare and being incredibly aggressive with my filtering but so many bots are ignoring robots.txt as well as lying about who they are with humanesque UAs that it’s having a real impact on my ability to provide the sites for humans.

    Over the past year it’s got around ten times worse. I woke up this morning to find my connection at a crawl and on checking the logs, AmazonBot has been hitting one site 12000 times an hour, and that’s one of the more well-behaved bots. But there’s thousands and thousands of them.


  • If cookie prompts annoy you (and why wouldn’t they? Complicated and time wasting prompts caused by terrible and compromised legislation that’s led to far more intrusion instead of enforcing use of browser settings) and you don’t care about cookies, then the browser extension “I don’t care about cookies” suppresses the vast majority.




  • I think this type of scheme is illegal under the GDPR, which is in effect in the UK just as it is in the EU.

    It’s been a while since I worked with the GDPR, but from memory the wording is such that:

    The data holder needs to allow people to opt out of data collection. The subject can request to be forgotten. The data holder explicitly cannot charge for this.

    But changes move slow, and The Mirror is probably banking on nobody caring enough to complain, and Trading Standards being too underfunded and swamped with other work to investigate otherwise (which they are). If they’re challenged, they’ll just change tack, go “oops” and are unlikely to hit big fines unless they dig in.

    Cookie laws are a horrible mess and always have done - the resulting consent banners are far more intrusive than anyone wanted.





  • Why would you want another year of their software for free?

    Because AV, like everything else, costs a fortune at enterprise scale.

    And yeah, I do understand your real point, but it’s really hard to choose good software. Every purchasing decision is a gamble and pretty much every time you choose something it’ll go bad sooner or later. (We didn’t imagine Vmware would turn into an extortion racket, for example. And we were only saying a few months ago how good value and reliable PRTG was, and they’ve just quadrupled their costs)

    It doesn’t matter how much due diligence and testing you put into software, it’s really hard to choose good stuff. Crowdstrike was the choice a year ago (the Linux thing was more recent than that), and its detection methods remain world class. Do we trust it? Hell no, but if we change to something else, there are risks and costs to that too.