{"id":360,"date":"2008-03-31T03:45:30","date_gmt":"2008-03-30T18:45:30","guid":{"rendered":"http:\/\/www.suzukikenichi.com\/blog\/google-webmaster-tools-robotstxt-generator\/"},"modified":"2008-03-31T03:49:24","modified_gmt":"2008-03-30T18:49:24","slug":"google-webmaster-tools-robotstxt-generator","status":"publish","type":"post","link":"https:\/\/www.suzukikenichi.com\/blog\/google-webmaster-tools-robotstxt-generator\/","title":{"rendered":"\u30b0\u30fc\u30b0\u30eb \u30a6\u30a7\u30d6\u30de\u30b9\u30bf\u30fc\u30c4\u30fc\u30eb\u306brobots.txt\u4f5c\u6210\u6a5f\u80fd\u304c\u8ffd\u52a0"},"content":{"rendered":"<p>Google \u30a6\u30a7\u30d6\u30de\u30b9\u30bf\u30fc\u30c4\u30fc\u30eb\u306brobots.txt\u3092\u4f5c\u6210\u3059\u308b\u6a5f\u80fd\u304c\u8ffd\u52a0\u3055\u308c\u307e\u3057\u305f\u3002<\/p>\n<p><a href=\"http:\/\/googlewebmastercentral.blogspot.com\/2008\/03\/speaking-language-of-robots.html\" rel=\"nofollow\">Google Webmaster Central Blog<\/a>\u3067\u3082\u516c\u5f0f\u306b\u30a2\u30ca\u30a6\u30f3\u30b9\u3055\u308c\u3066\u3044\u307e\u3059\u3002<br \/>\n<a href=\"http:\/\/www.flickr.com\/photos\/23318543@N07\/2371398197\/\"><br \/>\n<img decoding=\"async\" src=\"\/\/www.suzukikenichi.com\/images\/googlewebmastertoolsrobotstxt.jpg\" alt=\"google webmaster tools robots.txt\u751f\u6210\" \/><\/a><\/p>\n<p>robot.txt\u3068\u306f\u3001\u691c\u7d22\u30a8\u30f3\u30b8\u30f3\u306e\u30ed\u30dc\u30c3\u30c8\u3001\u3044\u308f\u3086\u308b\u30b9\u30d1\u30a4\u30c0\u30fc\u3082\u3057\u304f\u306f\u30af\u30ed\u30fc\u30e9\u30fc\u3068\u547c\u3070\u308c\u308b\u30d7\u30ed\u30b0\u30e9\u30e0\u306e\u30a2\u30af\u30bb\u30b9\u3092\u3001\u30b5\u30a4\u30c8\u5358\u4f4d\u30fb\u30c7\u30a3\u30ec\u30af\u30c8\u30ea\u5358\u4f4d\u30fb\u30d5\u30a1\u30a4\u30eb\u5358\u4f4d\u3067\u3001\u62d2\u5426\uff08\u8a31\u53ef\uff09\u3059\u308b\u305f\u3081\u306e\u30c6\u30ad\u30b9\u30c8\u30d5\u30a1\u30a4\u30eb\u3067\u3059\u3002<br \/>\n\u203b\u30b9\u30d1\u30a4\u30c0\u30fc\u3068\u30af\u30ed\u30fc\u30e9\u306f\u3001\u53b3\u5bc6\u306b\u306f\u7570\u306a\u308b\u30d7\u30ed\u30b0\u30e9\u30e0\u306e\u3088\u3046\u3067\u3059\u3002<br \/>\n\u25cf<a href=\"http:\/\/e-club3.hyperposition.com\/seoblog\/searchengine\/20070111095317.html\">\u691c\u7d22\u30a8\u30f3\u30b8\u30f3\u306e\uff15\u3064\u306e\u30d7\u30ed\u30b0\u30e9\u30e0 &#8211; \u30b9\u30d1\u30a4\u30c0\u30fc\u3068\u30af\u30ed\u30fc\u30e9\u3063\u3066\u5225\u7269\uff1f<\/a><br \/>\n\u25cf<a href=\"http:\/\/neoinspire.net\/archives\/85\">\u30b9\u30d1\u30a4\u30c0\u30fc\u3068\u30af\u30ed\u30fc\u30e9\u304c\u5225\u7269\u306a\u308f\u3051<\/a><\/p>\n<p>\u30a2\u30af\u30bb\u30b9\u3092\u30d6\u30ed\u30c3\u30af\u3059\u308b\u5fc5\u8981\u304c\u306a\u3051\u308c\u3070\u3001robots.txt\u306f\u7279\u306b\u5fc5\u8981\u3042\u308a\u307e\u305b\u3093\u3002<br \/>\n\u30a6\u30a7\u30d6\u30de\u30b9\u30bf\u30fc\u30c4\u30fc\u30eb\u3067\u306f\u3001\u300c\u3059\u3079\u3066\u306e\u30ed\u30dc\u30c3\u30c8\u3092\u8a31\u53ef\u300d\u3068\u3044\u3046\u6307\u5b9a\u3067robots.txt\u3092\u4f5c\u6210\u3067\u304d\u307e\u3059\u304c\u3001\u3059\u3079\u3066\u306e\u30ed\u30dc\u30c3\u30c8\u3092\u8a31\u53ef\uff08Allow\uff09\u3059\u308b\u306a\u3089\u3001\u4e0d\u8981\u3067\u3059\u3002<\/p>\n<p>\u4f5c\u308a\u65b9\u306f\u7c21\u5358\u3067\u3001Windows\u306b\u4ed8\u5c5e\u306e\u30e1\u30e2\u5e33\u306e\u3088\u3046\u306a\u30a8\u30c7\u30a3\u30bf\u3067\u4f5c\u308c\u307e\u3059\u3002<br \/>\n\u8a18\u8ff0\u306e\u66f8\u5f0f\u3082\u3044\u305f\u3063\u3066\u30b7\u30f3\u30d7\u30eb\u3067\u3059\u3002<\/p>\n<p>robots.txt\u306e\u66f8\u304d\u65b9\u306b\u3064\u3044\u3066\u306f\u3001\u30cd\u30c3\u30c8\u3067\u305f\u304f\u3055\u3093\u60c5\u5831\u304c\u5f97\u3089\u308c\u308b\u306e\u3067\u3053\u3053\u3067\u306f\u7701\u7565\u3057\u307e\u3059\u3002<br \/>\n\u300c<a href=\"http:\/\/www.google.co.jp\/search?hl=ja&#038;safe=off&#038;rls=GFRD%2CGFRD%3A2007-31%2CGFRD%3Aja&#038;q=robots.txt+%E6%9B%B8%E3%81%8D%E6%96%B9&#038;lr=\" rel=\"nofollow\">robots.txt \u66f8\u304d\u65b9<\/a>\u300d\u3084\u300c<a href=\"http:\/\/www.google.co.jp\/search?hl=ja&#038;safe=off&#038;rls=GFRD%2CGFRD%3A2007-31%2CGFRD%3Aja&#038;q=robots.txt+%E6%9B%B8%E5%BC%8F&#038;lr=\" rel=\"nofollow\">robots.txt \u66f8\u5f0f<\/a>\u300d\u3067\u691c\u7d22\u3057\u3066\u307f\u3066\u304f\u3060\u3055\u3044\u3002<br \/>\n\u203b\u5206\u304b\u3089\u306a\u3051\u308c\u3070\u3001\u30b3\u30e1\u30f3\u30c8\u3067\u8cea\u554f\u3092<\/p>\n<p>\u6163\u308c\u3066\u3057\u307e\u3048\u3070\u4f5c\u308b\u306e\u306f\u3069\u3046\u3063\u3066\u3053\u3068\u306e\u306a\u3044\u3082\u306e\u3060\u3057\u3001\u30d5\u30ea\u30fc\u306erobots.txt\u4f5c\u6210\u30c4\u30fc\u30eb\u3082\u51fa\u56de\u3063\u3066\u3044\u307e\u3059\u3002<\/p>\n<p>\u305d\u308c\u3067\u3082\u3001Google\u306e\u63d0\u4f9b\u3059\u308b\u30c4\u30fc\u30eb\u306e\u4e2d\u3067robots.txt\u3092\u4f5c\u308b\u3053\u3068\u304c\u3067\u304d\u308b\u306e\u306f\u4fbf\u5229\u306a\u3053\u3068\u3067\u3059\u3002<\/p>\n<p>Google \u30a6\u30a7\u30d6\u30de\u30b9\u30bf\u30fc\u30c4\u30fc\u30eb\u3067\u306erobots.txt\u751f\u6210\u6a5f\u80fd\u3067\u306f\u3001\u4ee5\u4e0b\u306e\uff15\u3064\u306e\u30ed\u30dc\u30c3\u30c8\u3092\u500b\u5225\u306b\u6307\u5b9a\u3059\u308b\u3053\u3068\u3082\u3067\u304d\u307e\u3059\u3002<\/p>\n<ul>\n<li>Googlebot <\/li>\n<li>Googlebot-Mobile <\/li>\n<li>Googlebot-Image <\/li>\n<li>Mediapartners-Google <\/li>\n<li>Adsbot-Google<\/li>\n<\/ul>\n<p><span style=\"font-weight:bold;\">Googlebot<\/span>\u306f\u901a\u5e38\u306e\u300c\u30a6\u30a7\u30d6\u691c\u7d22\u300d\u306e\u305f\u3081\u306e\u30ed\u30dc\u30c3\u30c8\u3001<span style=\"font-weight:bold;\">Google-Mobile<\/span>\u306f\u300c\u30e2\u30d0\u30a4\u30eb\u691c\u7d22\u300d\u306e\u305f\u3081\u306e\u30ed\u30dc\u30c3\u30c8\u3001<span style=\"font-weight:bold;\">Googlebot-Image<\/span>\u306f\u300c\u30a4\u30e1\u30fc\u30b8\u691c\u7d22\u300d\u306e\u305f\u3081\u306e\u30ed\u30dc\u30c3\u30c8\u3067\u3059\u3002<\/p>\n<p><span style=\"font-weight:bold;\">Mediapartners-Google<\/span>\u306f\u300cAdSense\u300d\u306e\u305f\u3081\u306e\u30ed\u30dc\u30c3\u30c8\u3067\u30da\u30fc\u30b8\u306e\u30b3\u30f3\u30c6\u30f3\u30c4\u306b\u3042\u3063\u305f\u5e83\u544a\u3092\u8abf\u3079\u308b\u305f\u3081\u306b\u4f7f\u308f\u308c\u3001<span style=\"font-weight:bold;\">Adsbot-Google<\/span>\u306f\u300cAdWords\u300d\u306e\u305f\u3081\u306e\u30ed\u30dc\u30c3\u30c8\u3067\u30a2\u30c9\u30ef\u30fc\u30ba\u5e83\u544a\u306e\u30ea\u30f3\u30af\u5148\u30da\u30fc\u30b8\u306e\u54c1\u8cea\u3092\u30c1\u30a7\u30c3\u30af\u3059\u308b\u305f\u3081\u306b\u4f7f\u308f\u308c\u307e\u3059\u3002<\/p>\n<p>\u6700\u5f8c\u306e\uff12\u3064\u306f\u62d2\u5426\u3059\u308b\u7406\u7531\u304c\u601d\u3044\u3064\u304d\u307e\u305b\u3093\u306d\u3002<\/p>\n<p>Mediapartners-Google\u3092\u30d6\u30ed\u30c3\u30af\u3059\u308b\u3068\u3069\u3093\u306a\u30a2\u30c9\u30bb\u30f3\u30b9\u5e83\u544a\u304c\u8868\u793a\u3055\u308c\u308b\u3093\u3067\u3057\u3087\u3046\uff1f<br \/>\n\u8ab0\u304b\u5b9f\u9a13\u3057\u3066\u307f\u3066\u304f\u308c\u307e\u305b\u3093\u304b\uff1f\uff08\u7b11\uff09<\/p>\n<p>Adsbot-Google\u3092\u30d6\u30ed\u30c3\u30af\u3059\u308b\u3068\u3001\u30e9\u30f3\u30c7\u30a3\u30f3\u30b0\u30da\u30fc\u30b8\u306e\u30af\u30aa\u30ea\u30c6\u30a3\u30c1\u30a7\u30c3\u30af\u304c\u3067\u304d\u306a\u3044\u306e\u3067\u30ad\u30fc\u30ef\u30fc\u30c9\u306e\u5165\u672d\u4fa1\u683c\u304c\u9ad8\u304f\u306a\u308b\u3093\u3067\u3057\u3087\u3046\u304b\uff1f<br \/>\n\u3053\u3061\u3089\u3082\u6050\u304f\u3066\u5b9f\u9a13\u3057\u3088\u3046\u3068\u306f\u601d\u3044\u307e\u305b\u3093\u3002\uff57<\/p>\n<p>\u3061\u306a\u307f\u306b\u3001\u4ed6\u306e\u691c\u7d22\u30a8\u30f3\u30b8\u30f3\u306e\u30ed\u30dc\u30c3\u30c8\u306e\u540d\u524d\u306f\u3069\u3046\u306a\u3063\u3066\u3044\u308b\u304b\u3068\u3044\u3046\u3068\u3001<br \/>\n\u30e4\u30d5\u30fc\u306f\u300c<span style=\"font-weight:bold;\">Slurp<\/span>\uff08\u30b9\u30e9\u30fc\u30d7\uff09\u300d\u3001MSN Live Search\u306f\u300c<span style=\"font-weight:bold;\">MSNbot<\/span>\uff08\u30a8\u30e0\u30a8\u30b9\u30a8\u30cc \u30dc\u30c3\u30c8\uff09\u300d\u3067\u3059\u3002<\/p>\n<p>\u3053\u308c\u3089\u306f\u3054\u5b58\u77e5\u3067\u3057\u3087\u3046\u3002<\/p>\n<p>\u30a2\u30e1\u30ea\u30ab\u3067\u306fGoogle\u3001Yahoo!\u3001MSN Live Search\u306b\u7d9a\u304f\u7b2c\uff14\u306e\u30b5\u30fc\u30c1\u30a8\u30f3\u30b8\u30f3\u306e<a href=\"http:\/\/www.ask.com\/\" rel=\"nofollow\">Ask.com<\/a>\uff08\u65e5\u672c\u3067\u306f<a href=\"http:\/\/ask.jp\/\" rel=\"nofollow\">Ask.jp<\/a>\uff09\u306f\u300c<span style=\"font-weight:bold;\">Teoma<\/span>\uff08\u30c6\u30aa\u30de\uff09\u300d\u3067\u3059\u3002<\/p>\n<p>\u904e\u53bb\u306e\u30a6\u30a7\u30d6\u30da\u30fc\u30b8\u3092\u4fdd\u5b58\u3059\u308b<a href=\"http:\/\/www.archive.org\/\" rel=\"nofollow\">Wayback Machine\u306eInternet Archive<\/a>\u306e\u30ed\u30dc\u30c3\u30c8\u306f\u3001\u300c<span style=\"font-weight:bold;\">ia_archiver<\/span>\uff08\u30a2\u30a4\u30a8\u30fc\u30fb\u30a2\u30fc\u30ab\u30a4\u30d0\u30fc\uff09\u300d\u3067\u3059\u3002<\/p>\n<p>Robot\uff08\u30ed\u30dc\u30c3\u30c8\uff09\u30fbSpider\uff08\u30b9\u30d1\u30a4\u30c0\u30fc\uff09\u30fbCrawler\uff08\u30af\u30ed\u30fc\u30e9\u30fc\uff09\u30fbBrowser\uff08\u30d6\u30e9\u30a6\u30b6\uff09\u3092\u307e\u3068\u3081\u3066\u3001User-Agent\uff08\u30e6\u30fc\u30b6\u30fc\u30fb\u30a8\u30fc\u30b8\u30a7\u30f3\u30c8\uff09\u3068\u3072\u3063\u304f\u308b\u3081\u308b\u3053\u3068\u3082\u3042\u308a\u307e\u3059\u3002<\/p>\n<p>User-Agent\u306e\u4e00\u89a7\u306f\u3001\u3053\u3061\u3089\u3067\u78ba\u8a8d\u3067\u304d\u307e\u3059\u3002<br \/>\n\u21d2 <a href=\"http:\/\/www.user-agents.org\/\">User-Agents.org<\/a><br \/>\n\u203b\u516c\u5f0f\u306e\u30b5\u30a4\u30c8\u3068\u3044\u3046\u308f\u3051\u3067\u306f\u306a\u3055\u305d\u3046\u3067\u3059\u3002<\/p>\n<p>\u7279\u5b9a\u306e\u30af\u30ed\u30fc\u30e9\u304b\u3089\u306e\u30a2\u30af\u30bb\u30b9\u3092\u62d2\u5426\uff08\u8a31\u53ef\uff09\u3057\u305f\u3044\u3068\u3044\u3046\u3068\u304d\u306b\u306f\u3001\u53c2\u7167\u3057\u3066\u304f\u3060\u3055\u3044\u3002<br \/>\n\u305f\u3060\u3057\u3001\u3059\u3079\u3066\u306e\u30af\u30ed\u30fc\u30e9\u304crobots.txt\u306b\u304a\u884c\u5100\u3088\u304f\u7d20\u76f4\u306b\u5f93\u3046\u3068\u3044\u3046\u308f\u3051\u3067\u3082\u306a\u3044\u306e\u3067\u3054\u6ce8\u610f\u3092\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Google \u30a6\u30a7\u30d6\u30de\u30b9\u30bf\u30fc\u30c4\u30fc\u30eb\u306brobots.txt\u3092\u4f5c\u6210\u3059\u308b\u6a5f\u80fd\u304c\u8ffd\u52a0\u3055\u308c\u307e\u3057\u305f\u3002 Google Webmaster Central Blog\u3067\u3082\u516c\u5f0f\u306b\u30a2\u30ca\u30a6\u30f3\u30b9\u3055\u308c\u3066\u3044\u307e\u3059\u3002 robot.txt\u3068\u306f\u3001\u691c\u7d22\u30a8\u30f3\u30b8\u30f3\u306e\u30ed\u30dc\u30c3\u30c8\u3001\u3044\u308f\u3086\u308b\u2026<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1,7],"tags":[18,23,24,28,54,62,80,81,116,340,341],"class_list":["post-360","post","type-post","status-publish","format-standard","category-google-seo","category-tools","tag-yahoo","tag-google","tag-live-search","tag-msn","tag-seo-tools","tag-robotstxt","tag-crawler","tag-googlebot","tag-116","tag-slurp","tag-msnbot"],"_links":{"self":[{"href":"https:\/\/www.suzukikenichi.com\/blog\/wp-json\/wp\/v2\/posts\/360","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.suzukikenichi.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.suzukikenichi.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.suzukikenichi.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.suzukikenichi.com\/blog\/wp-json\/wp\/v2\/comments?post=360"}],"version-history":[{"count":0,"href":"https:\/\/www.suzukikenichi.com\/blog\/wp-json\/wp\/v2\/posts\/360\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.suzukikenichi.com\/blog\/wp-json\/wp\/v2\/media?parent=360"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.suzukikenichi.com\/blog\/wp-json\/wp\/v2\/categories?post=360"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.suzukikenichi.com\/blog\/wp-json\/wp\/v2\/tags?post=360"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}