# Disallow all crawlers access to certain pages. # User-agent: * ��04/10/31 for google Adsense User-agent: * disallow: /CACHE/ disallow: /copies/ disallow: /host/ disallow: /kiri_/ disallow: /nano/ disallow: /olive/ disallow: /test/ disallow: /post/ disallow: /yui_/ user-agent: * disallow: / user-agent: Baiduspider disallow: / user-agent: BaiduImagespider disallow: / user-agent: BaiduMobaider disallow: / User-Agent: DoCoMo/2.0 P05A(c100;TB;W24H15) (compatible; BaiduMobaider/1.0; +http://www.baidu.jp/spider/) disallow: / user-agent: YodaoBot disallow: /