User-agent: * Crawl-Delay: 30 Disallow: /bd_mobile/ Disallow: /usaepay/ Disallow: /support/ Disallow: /bd2016/ Allow: / sitemap:http://www.beverlydiamonds.com/sitemap.xml User-agent: Baiduspider Disallow: / User-agent: baidu Disallow: / User-agent: sogou Disallow: / User-agent: Yandex Disallow: / User-agent: aranhabot Disallow: / User-agent: Yahoo! Slurp Disallow: / User-agent: MJ12bot Disallow: / User-agent: LexxeBot/1.0 Disallow: / User-Agent: oBot Disallow: / User-Agent: SputnikBot Disallow: / User-agent: Symfony Spider Disallow: / User-Agent: oBot Disallow: / User-Agent: SputnikBot Disallow: / User-agent: Symfony Spider Disallow: / User-agent: EasouSpider Disallow: / User-agent: panscient.com Disallow: / User-agent: LexxeBot/1.0 Disallow: / User-agent: NextGenSearchBot Disallow: / User-agent: Sosospider Disallow: / User-agent: Sosospider+(+http://help.soso.com/webspider.htm) Disallow: / User-agent: rogerbot Disallow: / User-agent: SiteBot/0.1 Disallow: / User-agent: SiteBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: Baiduspider Disallow: / User-agent: crystalsemanticsbot Disallow: / User-agent: CrystalSemanticsBot Disallow: / User-agent: NetSeer crawler Disallow: / User-agent: trovitBot Disallow: / User-agent: LexxeBot Disallow: / User-agent: DotBot Disallow: / User-agent: Ezooms Disallow: / User-agent: discobot Disallow: / User-agent: Jyxobot Disallow: / User-agent: sogou Disallow: / User-agent: sogou spider Disallow: / User-agent: Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07) Disallow: / User-agent: sistrix Disallow: / User-agent: heritrix Disallow: / User-agent: GarlikCrawler/1.1 (http://garlik.com/, crawler@garlik.com) Disallow: / User-agent: AhrefsBot Disallow: / User-agent: NerdByNature.Bot Disallow: / User-agent: psbot Disallow: / User-agent: WBSearchBot Disallow: / User-agent: AddThis.com Disallow: / User-agent: ia_archiver Disallow: / User-agent: proximic Disallow: / User-agent: discoverybot Disallow: / User-agent: bl.uk_lddc_bot Disallow: / User-agent: IstellaBot Disallow: / User-agent: seokicks Disallow: / User-agent: SEOkicks-Robot Disallow: / User-agent: UnisterBot Disallow: / User-agent: Bender Disallow: / User-agent: wotbox Disallow: / User-agent: Yasni Disallow: / User-agent: JikeSpider Disallow: / User-agent: netEstate NE Crawler Disallow: / User-agent: Exabot Disallow: / User-agent: Pixray-Seeker Disallow: / User-agent: Linguee Disallow: / User-agent: integromedb Disallow: / User-agent: SearchmetricsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: BDCbot Disallow: / User-agent: GrapeshotCrawler Disallow: / User-agent: WeSEE:Search Disallow: / User-agent: TurnitinBot Disallow: / User-agent: admantx Disallow: / User-agent: spbot Disallow: / User-agent: discobot Disallow: / User-agent: AhrefsBot disallow: / User-agent: Mail.RU_Bot disallow: /