🌍
English
Googlebot is doing 1000 hits an hour. Do you have some code to let them crawl my website?
The site has been online for 2 days and on a shared hosting plan. How do I block it?
Hi,
a little search on Google (yes !) and you could find that http://www.robotstxt.org/robotstxt.html
if you want to completely block Googlebot, create a robots.txt file with this content
User-agent: Google Disallow: /
but your website won't be available on Google search engine
Offline