User-agent: * Disallow: /Admin Disallow: /siteadmin Disallow: /swf Disallow: /archief #idiot-bot; generates tons of 404 and malformed urls User-agent: wget Disallow: / User-agent: ExtractorPro Disallow: /