[14.0][ADD] website_no_crawler_environment: initial commit #891
Conversation
| <xpath expr="t" position="replace"> | ||
| <t t-translation="off"> | ||
| User-agent: * | ||
| <t t-if="server_env in ('preprod', 'stage')">Disallow: /</t> |
There was a problem hiding this comment.
In this way you're limiting user to have preprod and stage env only. Better to generalize it, having test as default one.
|
@AshishHirapara Better to move to https://github.com/OCA/server-env repo |
| Sitemap: | ||
| <t t-esc="url_root" /> | ||
| sitemap.xml |
There was a problem hiding this comment.
where is sitemap.xml?
What are the advantages of this module over using X-Robots-Tag?
There was a problem hiding this comment.
@Tardo I think the main advantage of this module is that users don't need to have knowledge of 'developer-specific' and complicated things like the X-Robots-Tag, and many users don't even know how to use the X-Robots-Tag within the odoo, so the module will be useful for them.
Also, this module provides a way to overwrite the view that generates /robots.txt and won't allow to index based on the server environment, so it will be easy to use if users have multiple environments and just restrict some of the environments from crawlers. So this is the main purpose of this module.
And if you think this can be done in a better way with X-Robots-Tag, then I am open to your suggestions. Let me know.
Thanks.
|
There hasn't been any activity on this pull request in the past 4 months, so it has been marked as stale and it will be closed automatically if no further activity occurs in the next 30 days. |
github-actions
bot
added
the
stale
By default, /robots.txt on an Odoo installation with a website module installed will allow indexing by web crawlers.
This module will overwrite the view that generates /robots.txt and won't allow index based on the server environment.