john pfeiffer
  • Home
  • Categories
  • Tags
  • Archives

drupal controlling web crawlers google results

drupal-controlling-web-crawlers-google-results

Google search for site:yourdomain.com and you'll see what Google has cached/stored as results.

If you end up with hundreds of strange results from Google

leading to pages like: /en/admin/reports/access/13745 /bg/admin/reports/access/13745

FIRST, user permissions -> anonymous users should not have access to:

statistics module -> access statistics

(But you may want them to see "view post access counter" to see how often a post has been read)

ALSO, the default /drupalroot/robots.txt does not cover scenarios with the leading language...

you must manually update robots.txt

Disallow: /admin/ Disallow: /bg/admin/ Disallow: /en/admin/


  • « BAT strings date improved script silent install
  • wss 3 style modifications »

Published

Feb 8, 2011

Category

drupal

~90 words

Tags

  • controlling 1
  • crawlers 1
  • drupal 43
  • google 18
  • results 1
  • web 56