robots文件
拿去用User-agent: Baiduspider
Allow: /$
Disallow: /*
User-agent: Baiduspider-image
Allow: /$
Disallow: /*
User-agent: Baiduspider-news
Allow: /$
Disallow: /*
User-agent: Googlebot
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*
User-agent: MSNBot
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*
User-agent: YoudaoBot
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*
User-agent: Sogou web spider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*
User-agent: Sogou inst spider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*
User-agent: Sogou spider2
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*
User-agent: Sogou blog
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*
User-agent: Sogou News Spider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*
User-agent: Sogou Orion spider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*
User-agent: ChinasoSpider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*
User-agent: Sosospider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*
User-agent: yisouspider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*
User-agent: EasouSpider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*
User-agent: 360Spider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*
User-Agent: *
Disallow: /
复制代码
页:
[1]