Only Index Docs from Currently Supported Versions

[dave2wave]

In Algolia there is a crawler configuration which skips "incubating" docs. I think that we should stop creating an index for versions of pulsar components which are no longer supporting.

(1) We can either remove these pages from the site.

or more likely

(2) Include those versions in the exclude pattern.

[michaeljmarshall]

(2) Include those versions in the exclude pattern.

I support this option--I assume that some users are still running outdated versions of Pulsar, and they would benefit from the docs being online.

[tisonkun]

I agree. And leave the docs of old versions as is.

I talked with @momo-jun today concluding a similar result. Coming up with a proposal in weeks.

[github-actions]

The issue had no activity for 30 days, mark with Stale label.

[tisonkun]

FYI @urfreespace

[tisonkun]

Closed as done. The incubating pages are removed from the page.

About other unmaintained docs pages, we may later come up with another proposal.

@dave2wave How does Algolia works to exclude those unmaintained docs? I need some technical income to find a way to continue this cleanup work.

[tisonkun]

I notice that the version config contains a option noIndex https://docusaurus.io/docs/api/plugins/@docusaurus/plugin-content-docs#VersionsConfig to hint the search engine index those version, will investigate.

[tisonkun]

Delivered noIndex config at apache/pulsar-site#506. But I'm not sure if Algolia omit ancient versions now or we should still make more configs.

[tisonkun]

Closed as somehow we now have only indices for new versions. I'm not sure why but let's open a new issue if there is any regressions.

[tisonkun]

Maybe the noIndex metadata helps.