Announcement

#1 2013-02-14 08:59:09

shaun
Guest

Goolglebot

Googlebot is doing 1000 hits an hour.  Do you have some code to let them crawl my website?

The site has been online for 2 days and on a shared hosting plan.  How do I block it?

 

#2 2013-02-14 10:04:10

mistic100
Former Piwigo Team
Lyon (FR)
2008-09-27
3277

Re: Goolglebot

Hi,

a little search on Google (yes !) and you could find that http://www.robotstxt.org/robotstxt.html

if you want to completely block Googlebot, create a robots.txt file with this content

Code:

User-agent: Google
Disallow: /

but your website won't be available on Google search engine

Offline

 

Board footer

Powered by FluxBB

github twitter newsletter Donate Piwigo.org © 2002-2024 · Contact