New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

With defaultContentLanguageInSubdir: true, a robots.txt file should be created at the root #5160

Open
martignoni opened this Issue Sep 4, 2018 · 3 comments

Comments

Projects
None yet
3 participants
@martignoni
Copy link

martignoni commented Sep 4, 2018

When option defaultContentLanguageInSubdiris set to true, a sitemap.xml file is automagically generated by Hugo at the root of the website, as well as in all language subdirectories.

I propose that the same is done for the robots.txt file, which is currently (as of Hugo 0.48) only generated in language subdirectories but not at the root of the website.

Rationale:

  • SEO tools don't work well without a robots.txt file in the root of a website.
  • Consistency with sitemap.xml behaviour.

See: https://discourse.gohugo.io/t/multilingual-site-always-get-a-noindex-tag-in-root-index-html

@divinerites

This comment has been minimized.

Copy link

divinerites commented Sep 4, 2018

Thanks a lot @martignoni.

And I suspect this will lead to the same issue for 404 page (see https://discourse.gohugo.io/t/404-html-not-generated-on-root-if-defaultcontentlanguageinsubdir-true/14048).

A bundle 😸

EDIT / Done with #5161

martignoni added a commit to moodlebox/moodlebox.net that referenced this issue Oct 27, 2018

Add robots.txt to site root
- Workaround for gohugoio/hugo#5160.
- To be deleted when above issue is resolved.
@stale

This comment has been minimized.

Copy link

stale bot commented Jan 3, 2019

This issue has been automatically marked as stale because it has not had recent activity. The resources of the Hugo team are limited, and so we are asking for your help.
If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open.
If this is a feature request, and you feel that it is still relevant and valuable, please tell us why.
This issue will automatically be closed in the near future if no further activity occurs. Thank you for all your contributions.

@stale stale bot added the Stale label Jan 3, 2019

@martignoni

This comment has been minimized.

Copy link
Author

martignoni commented Jan 3, 2019

Still present as of Hugo 0.53.

@stale stale bot removed the Stale label Jan 3, 2019

@moorereason moorereason added the Keep label Jan 3, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment