# Robots.txt # Allow all robots to spider the domain User-agent: * Disallow: # Disallow directory /_assets/server/ User-agent: * Disallow: /_assets/server/ Disallow: /blog/ # Disallow any backup files Disallow: /index copy (06-05-09).asp