You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What steps will reproduce the problem?
1. put Disallow: /example/ in robots.txt
2. or put /example/ in Excluded URL patterns
3. what it works if you put /example/* in Excluded URL patterns
What is the expected output? What do you see instead?
it is expected that the directory /example/ to be ingnored and excluded in
the sitemap
What version of the product are you using? On what operating system?
sitemap_linux-i386-beta1.tar.gz
Please provide any additional information below.
tested on two servers ubuntu 8.10 intrepid also on jaunty
also seems that robots.txt is parsed (i watched the logs)
Original issue reported on code.google.com by map...@gmail.com on 23 Feb 2009 at 10:14
The text was updated successfully, but these errors were encountered:
Just want to make things clear:
>> 1. put Disallow: /example/ in robots.txt
If /example/some-page appears before robots.txt is updated. This has no effect.
>> 2. or put /example/ in Excluded URL patterns
Yes, this is an expected behavior.
>> 3. what it works if you put /example/* in Excluded URL patterns
This works. Right?
Original comment by ma...@google.com on 25 Feb 2009 at 5:27
with latest beta i check now if it works
1) point seems to me that robots.txt rules are now working /example/
and /example/2 are now ignored and not added in latest beta (from 25 Feb)
2) OK but i thought that the same rules from robots.txt are applied
so it should be more consistent to apply the same rules as in robots.txt
/example/ to be like /example/*
3. yes i was just notice you that i was expecting /example/ to ignore all
subdirs
/example/2 ...
Original comment by map...@gmail.com on 26 Feb 2009 at 9:23
Experiencing the same problem: /eDM/ and /WDSearch/ are both disallowed in
robots.txt. Configured GSG to exclude /eDM/* and /WDSearch/* and URLs still
included. Configured GSG to exclude /eDM/ and /WDSearch/ and URLs still
included.
Using sitemap_win-beta1-20091231.msi on Windows 2003
Original comment by johnlgoo...@gmail.com on 5 Mar 2010 at 10:39
Original issue reported on code.google.com by
map...@gmail.com
on 23 Feb 2009 at 10:14The text was updated successfully, but these errors were encountered: