User-agent: * Disallow: /cgi-bin Disallow: /scripts Disallow: /alogreports Disallow: /forum Disallow: /testing Disallow: /pics Disallow: /common Disallow: /images #idiot-bot; generates tons of 404 and malformed urls User-agent: wget Disallow: /