BlueKey

The hidden services bother with a robots.txt file as much as the EU crawlers.
They can just start at an Onion link collection and start crawling.

I’m a bit confused.
Will this affect only the home feed or also the recommendations beside a playing video?

Don’t you see how this improves law-enforcement? Now the criminals won’t dare to do forbidden stuff there.
-- some government, somewhere