robots.txt 禁止抓取标签问题 #164

Closed
figo930 opened this Issue Jan 16, 2013 · 1 comment

Projects

None yet

2 participants

@figo930
figo930 commented Jan 16, 2013

User-agent: *
Allow: /blog-articles-feed.do
Allow: /tag-articles-feed.do*
Disallow: /.do
Disallow: /login
Disallow: /logout
Disallow: /tags
Disallow: /authors/

Disallow: /console/*
Disallow: /error/*

规则里 Disallow: /tags 直接禁止了所有搜索引擎对 tags 标签的抓取,建议删掉

@88250 88250 was assigned Jan 17, 2013
@88250
Member
88250 commented Jan 17, 2013

嗯,0.5.6 修复。

多谢 👍

@88250 88250 added a commit that closed this issue Jan 17, 2013
@88250 88250 Fixed #164 989c6df
@88250 88250 closed this in 989c6df Jan 17, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment