2008
The Web Robots Pages
The Robots Database lists robot software implementations and operators.
2006
2005
Database of Web Robots | robotstxt.org
liste détaillée des différents robots surfeurs existants
Robots.txt Generator | mcanerin.com
by 2 othersCreate Robots.txt to allow/disallow search crawlers in certain directories on your site
1
(4 marks)