老榕树社区

标题: robots文件 [打印本页]

作者: yoki343 时间: 2017-3-13 11:12
标题: robots文件
拿去用

User-agent: Baiduspider
Allow: /$
Disallow: /*

User-agent: Baiduspider-image
Allow: /$
Disallow: /*

User-agent: Baiduspider-news
Allow: /$
Disallow: /*

User-agent: Googlebot
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*

User-agent: MSNBot
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*

User-agent: YoudaoBot
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*

User-agent: Sogou web spider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*

User-agent: Sogou inst spider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*

User-agent: Sogou spider2
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*

User-agent: Sogou blog
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*

User-agent: Sogou News Spider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*

User-agent: Sogou Orion spider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*

User-agent: ChinasoSpider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*

User-agent: Sosospider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*

User-agent: yisouspider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*

User-agent: EasouSpider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*

User-agent: 360Spider
Disallow: /user/
Disallow: /group/
Disallow: /?*
Disallow: /*?*

User-Agent: *
Disallow: /
复制代码

欢迎光临老榕树社区 (http://club.lrswl.com/)