Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Compiler] Add sitemap to doc generator #8348

Merged

Conversation

@straight-shoota
Copy link
Member

straight-shoota commented Oct 18, 2019

The doc generator can produce a sitemap.xml which lists all HTML pages accessible for search engines. The goal is to use this sitemap to assign lower priorities to outdated doc pages on https://crystal-lang.org/api/ to make the most recent release (latest) show up first in search results. See crystal-lang/crystal-website#79 for details. Having such a sitemap is generally useful for shards as well, so it makes sense to directly integrate it into the docs generator.

Sitemaps are only generated when --sitemap-base-url=URL option is set. Sitemaps require absolute paths, so it can't be produced without a base url. Additional options are --sitemap-priority and --sitemap-changefreq which correspond do the respective sitemap features.

This also reverts #5990 which tried an alternative approach to solving the search priority issue using canonical URLs. But this completely removes older versions from search results.

Ideally, sitemap.xml should be compressed, but the compiler is supposed to be built without libz, so this needs to be done separately.

This reverts commit a6d0571.
@straight-shoota straight-shoota force-pushed the straight-shoota:feature/docs-sitemap branch from dfb4071 to 236e120 Oct 29, 2019
@RX14
RX14 approved these changes Oct 29, 2019
Makefile Outdated Show resolved Hide resolved
@straight-shoota straight-shoota force-pushed the straight-shoota:feature/docs-sitemap branch from 236e120 to c809501 Oct 29, 2019
This is useful for specs where all the configuration options are
irrelevant.
@straight-shoota straight-shoota force-pushed the straight-shoota:feature/docs-sitemap branch from e781fc4 to 6536e81 Oct 29, 2019
@straight-shoota straight-shoota force-pushed the straight-shoota:feature/docs-sitemap branch from 6536e81 to 3e07569 Oct 30, 2019
@straight-shoota straight-shoota merged commit 42bab8d into crystal-lang:master Oct 30, 2019
6 checks passed
6 checks passed
ci/circleci: check_format Your tests passed on CircleCI!
Details
ci/circleci: test_darwin Your tests passed on CircleCI!
Details
ci/circleci: test_linux Your tests passed on CircleCI!
Details
ci/circleci: test_linux32_std Your tests passed on CircleCI!
Details
ci/circleci: test_preview_mt Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@straight-shoota straight-shoota deleted the straight-shoota:feature/docs-sitemap branch Oct 30, 2019
@bcardiff bcardiff added this to the 0.32.0 milestone Dec 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.