Cross-repo link checking not working as expected

[colleenmcginnis]

Describe the bug

We've seen a couple instances of cross-repo link checking not working as expected.

Example 1: Missing ID

• Before Align Observability logs docs docs-content#547 was merged, there was a log-correlation-intro ID in the solutions/observability/logs/stream-application-logs.md file in the elastic/docs-content repo.
• In elastic/asciidocalypse, there was a cross-repo link to this ID.
• In Align Observability logs docs docs-content#547, the log-correlation-intro ID was removed, but the build succeeded.
  • 💡 Is this because it was checking cross-repo links to docs-content against main instead of the working branch?
• Align Observability logs docs docs-content#547 is merged.
• PRs previews in elastic/asciidocalypse start failing because there is a cross-repo link to a now non-existent ID (doesn't exist on main).
• We fixed the error in elastic/asciidocalypse in https://github.com/elastic/asciidocalypse/pull/118.

Example 2: Missing file

• Before https://github.com/elastic/asciidocalypse/pull/116 was merged, the first page of the integration docs was located at integration-docs/docs/reference/ingestion-tools/index.md in the elastic/asciidocalypse repo.
• In elastic/docs-content, there were two cross-repo links (one, two) to this file.
• In https://github.com/elastic/asciidocalypse/pull/116, the file was renamed from integration-docs/docs/reference/ingestion-tools/index.md to integration-docs/docs/reference/ingestion-tools/integrations/index.md, but the build succeeded.
• Even though the check passed, I anticipated this would cause a failure in docs-content the next time a preview build started so I added a redirect.
• I think this prevented errors in docs-content.

Expected behavior

The build should have failed in the two examples above.

Steps to reproduce

1. Find a cross-repo link.
2. Go to the repo that contains the file you're linking to.
3. Delete that file.
4. Open a PR with that change.
5. The check will pass even though the link will be broken after the PR is merged.

Tooling

• docs-builder
• migration tooling
• I'm not sure

[Mpdreamz]

The docs-builder today only validates outbound links.

• repo A has file-a.md

• repo B has file-b.md

• repo B file-b.md links to a://file-a.md

If repo A deletes file-a.md docs-builder only validates the current content set of A has no active links to file-a.md. We further only validate that the outbound cross links from A to other docsets are valid.

We will have a second PR github action that will validate incomming links by looking for file-a.md usages in other docsets.

Each links.json already publishes all external cross links in use by that particular docset.

To aid with this we already have a cron job that publishes a link-index.json that points to all known docset's link.json files. This is done from this repository https://github.com/elastic/docs-internal-workflows as an FYI for the curious.