User-agent: China Local Browse 2.6 Disallow: / User-agent: ContactBot/0.2 Disallow: / User-agent: ContentSmartz Disallow: / User-agent: DBrowse 1.4b Disallow: / User-agent: DBrowse 1.4d Disallow: / User-agent: DSurf15a 01 Disallow: / User-agent: DSurf15a 71 Disallow: / User-agent: DSurf15a 81 Disallow: / User-agent: DSurf15a VA Disallow: / User-agent: DataCha0s/2.0 Disallow: / User-agent: EBrowse 1.4b Disallow: / User-agent: ESurf15a 15 Disallow: / User-agent: Educate Search VxB Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailSpider Disallow: / User-agent: EmailWolf 1.00 Disallow: / User-agent: ExtractorPro Disallow: / User-agent: FSurf15a 01 Disallow: / User-agent: Franklin Locator 1.8 Disallow: / User-agent: Full Web Bot 0416B Disallow: / User-agent: Full Web Bot 0516B Disallow: / User-agent: Full Web Bot 2816B Disallow: / User-agent: Googlebot Disallow: / User-agent: Mediapartners-Google Disallow: / User-agent: Adsbot-Google Disallow: / User-agent: Guestbook Auto Submitter Disallow: / User-agent: IUPUI Research Bot v 1.9a Disallow: / User-agent: Industry Program 1.0.x Disallow: / User-agent: Jeeves Disallow: / User-agent: LMQueueBot/0.2 Disallow: / User-agent: LWP::Simple/5.803 Disallow: / User-agent: LetsCrawl.com/1.0 +http://letscrawl.com/ Disallow: / User-agent: MFC Foundation Class Library 4.0 Disallow: / User-agent: MVAClient Disallow: / User-agent: Mac Finder 1.0.xx Disallow: / User-agent: Microsoft URL Control - 6.00.8xxx Disallow: / User-agent: Missauga Locate 1.0.0 Disallow: / User-agent: Missigua Locator 1.9 Disallow: / User-agent: Missouri College Browse Disallow: / User-agent: Mizzu Labs 2.2 Disallow: / User-agent: NameOfAgent (CMS Spider) Disallow: / User-agent: Nsauditor/1.x Disallow: / User-agent: PBrowse 1.4b Disallow: / User-agent: PEval 1.4b Disallow: / User-agent: PSurf15a 11 Disallow: / User-agent: PSurf15a 51 Disallow: / User-agent: PSurf15a VA Disallow: / User-agent: Port Huron Labs Disallow: / User-agent: Production Bot 0116B Disallow: / User-agent: Production Bot 2016B Disallow: / User-agent: Production Bot DOT 3016B Disallow: / User-agent: Program Shareware 1.0.2 Disallow: / User-agent: RSurf15a 41 Disallow: / User-agent: RSurf15a 51 Disallow: / User-agent: RSurf15a 81 Disallow: / User-agent: SC Systems iRc Search 2.1 Disallow: / User-agent: SSurf15a 11 Disallow: / User-agent: ShablastBot 1.0 Disallow: / User-agent: Slurp Disallow: / User-agent: Yahoo-MMCrawler Disallow: / User-agent: Snapbot/1.0 Disallow: / User-agent: Snapbot/1.0 (Snap Shots, +http://www.snap.com) Disallow: / User-agent: Sogou Orion spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07) Disallow: / User-agent: Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07) Disallow: / User-agent: TSurf15a 11 Disallow: / User-agent: Under the Rainbow 2.2 Disallow: / User-agent: VadixBot Disallow: / User-agent: WEP Search 00 Disallow: / User-agent: Wells Search II Disallow: / User-agent: atSpider/1.0 Disallow: / User-agent: autoemailspider Disallow: / User-agent: bwh3_user_agent Disallow: / User-agent: msnbot Disallow: / User-agent: psbot Disallow: / User-agent: psycheclone Disallow: / User-agent: searchbot admin@google.com Disallow: / User-agent: snap.com beta crawler v0 Disallow: / User-agent: sogou develop spider Disallow: / User-agent: sogou spider Disallow: / User-agent: sohu agent Disallow: / User-agent: * Crawl-delay: 960.00 Disallow: / Disallow: /*.gif$ Disallow: /*.Gif$ Disallow: /*.GIF$ Disallow: /*.jpg$ Disallow: /*.Jpg$ Disallow: /*.JPG$ Disallow: /*.jpeg$ Disallow: /*.Jpeg$ Disallow: /*.JPEG$ Disallow: /*.js$ Disallow: /*.pdf$ Disallow: /*.Pdf$ Disallow: /*.PDF$ Disallow: /*.php$ Disallow: /*.png$ Disallow: /*.zip$ Disallow: /*.Zip$ Disallow: /*.ZIP$