Mostly work related stuff which I would've entered into my "Log book". Instead of hosting it on an intranet site, outsourcing the hosting to blogger!
https://stackoverflow.com/questions/56049660/how-to-exclude-all-robots-except-googlebot-and-bingbot-with-both-robots-txt-and
So for allowing googlebot and bingbot and nothing else, the syntax would be:
User-agent: *Disallow: /
User-agent: BingbotDisallow:
User-agent: GooglebotDisallow:
No comments:
Post a Comment