Migrate validation to Jekyll hooks

[marcwrobel]

Compared to the use of the validate.py script / validate.yaml workflow, this has the advantage of ensuring the validation is always performed. It also gives access to enriched data, which is useful for validated generated content, such as links.

In addition to checks already implemented in the previous validate.py script, this script:

• Check that release dates are not too far in the future to avoid mistakes such as [semeru] Fix latestreleaseDate for v8 #3281 (max 30 days allowed).
• Check all links are valid (even those in the products description). This check takes time and is not performed by default, but it can be triggered by setting the MUST_CHECK_URLS environment variable to true. To ensure links are regularly verified a new GitHub workflow, check-links.yml, has also been added. It runs every Sunday at midnight.

Note that:

• Some links had to be excluded from the link validation because, despite being valid, they are sometime or systematically returning an error.
• A dependency to open-uri has been added to simplify the validation of HTTP links.
• The priority of the ProductDataEnricher plugin had to be set to normal, to allow hooks to be run before and after enrichment.
• The deploy.sh has been cleaned-up and enriched with the deploy URL replacement previously done in netlify.toml.

Some alternate URLs has been changed or removed due to an enforcement of the rules regarding permalinks and alternate_urls (no more dots).

Closes #1206, #2709, #3409.

[marcwrobel]

Adding a workflow to validate URLs every week. Converting this PR to draft.

[marcwrobel]

@captn3m0, I just added a new workflow to trigger the links verification. You can see the result on https://github.com/endoflife-date/endoflife.date/actions/runs/5098680434/jobs/9165967471.

[marcwrobel]

Oddly the deploy-preview does not fail when there are errors. Will have to reinstate the validate.yml workflow.

[marcwrobel]

Oddly the deploy-preview does not fail when there are errors. Will have to reinstate the validate.yml workflow.

I found why : the error I introduced in one of the releaseDate of Alibaba DragonWell was overwritten by the latest.py script launched by the deploy script. Now I have the following error as expected and the Netlify deploy task fails with an error:

10:33:30 AM:  Incremental build: disabled. Enable with --incremental
10:33:30 AM:       Generating...
10:33:30 AM:  Product Validator: Invalid releasePolicyLink 'N/A' for graylog.md, should match (?-mix:^https?:\/\/.+$).
10:33:31 AM:        Jekyll Feed: Generating feed for posts
10:34:29 AM: /opt/build/repo/_plugins/product-data-validator.rb:317:in `block in <top (required)>': Site build canceled : 1 errors detected (RuntimeError)

[marcwrobel]

Re-checked validation errors:

• 🆗 Invalid title '12' for coldfusion.md, expecting a value of type String, got Integer.
• 🆗 Invalid category 'server-app2' for coldfusion.md, expecting one of app, db, device, framework, lang, library, os, server-app, service, standard.
• 🆗 Invalid tags 'adobe!' for coldfusion.md, should match (?-mix:^[a-z0-9\-]+( [a-z0-9\-]+)*$).
• 🆗 Invalid permalink 'coldfusion' for coldfusion.md, should match (?-mix:^\/[a-z0-9-]+$).
• 🆗 Invalid alternate_urls 'cold' for coldfusion.md, should match (?-mix:^\/[a-z0-9\-_]+$).
• 🆗 Invalid versionCommand 'true' for coldfusion.md, expecting a value of type String, got TrueClass.
• 🆗 Invalid releasePolicyLink 'https:/invalid' for coldfusion.md, should match (?-mix:^https?:\/\/.+$).
• 🆗 Invalid changelogTemplate 'ftp://invalid' for coldfusion.md, should match (?-mix:^https?:\/\/.+$).
• 🆗 Invalid releaseImage 'invalid link' for coldfusion.md, should match (?-mix:^https?:\/\/.+$).
• 🆗 Invalid releaseLabel 'true' for coldfusion.md, expecting a value of type String, got TrueClass.
• 🆗 Invalid LTSLabel 'true' for coldfusion.md, expecting a value of type String, got TrueClass.
• 🆗 Invalid eolColumn '12' for coldfusion.md, expecting a value of type boolean or string, got Integer.
• 🆗 Invalid activeSupportColumn '12' for coldfusion.md, expecting a value of type boolean or string, got Integer.
• 🆗 Invalid releaseColumn '12' for coldfusion.md, expecting a value of type boolean or string, got Integer.
• 🆗 Invalid releaseDateColumn '12' for coldfusion.md, expecting a value of type boolean or string, got Integer.
• 🆗 Invalid extendedSupportColumn '12' for coldfusion.md, expecting a value of type boolean or string, got Integer.
• 🆗 Invalid eolWarnThreshold 'a' for coldfusion.md, expecting a value of type numeric, got String.
• 🆗 Invalid activeSupportWarnThreshold 'a' for coldfusion.md, expecting a value of type numeric, got String.
• 🆗 Invalid discontinuedColumn '12' for coldfusion.md, expecting a value of type boolean or string, got Integer.
• 🆗 Invalid discontinuedWarnThreshold 'a' for coldfusion.md, expecting a value of type numeric, got String.
• 🆗 Invalid extendedSupportWarnThreshold 'a' for coldfusion.md, expecting a value of type numeric, got String.
• 🆗 Invalid auto 'git' for coldfusion.md, expecting and Array, got String.
• 🆗 Invalid identifiers 'abc' for coldfusion.md, expecting and Array, got String.
• 🆗 Invalid releaseCycle '2023' for coldfusion.md#2023, expecting a value of type String, got Integer.
• 🆗 Invalid releaseDate 'aaa' for coldfusion.md#2023, expecting a value of type boolean or date, got String.
• 🆗 Invalid eol 'aaa' for coldfusion.md#2023, expecting a value of type boolean or date, got String.
• 🆗 Invalid extendedSupport 'aaa' for coldfusion.md#2023, expecting a value of type boolean or date, got String.
• 🆗 Invalid codename '2023' for coldfusion.md#2023, expecting a value of type String, got Integer.
• 🆗 Invalid releaseDate '2027-05-17' for coldfusion.md#2023, expecting a value in the next 30 days, got 2027-05-17.
• 🆗 Invalid lts 'blabla' for coldfusion.md#2023, expecting a value of type boolean or date, got String.
• 🆗 Invalid latest 'true' for coldfusion.md#2023, expecting a value of type String, got TrueClass.
• 🆗 Invalid latest '' for coldfusion.md#2021, expecting a value of type String, got NilClass.
• 🆗 Invalid latestReleaseDate 'aaa' for coldfusion.md#2023, expecting a value of type boolean or date, got String.
• 🆗 Invalid link 'toto' for coldfusion.md#2023, should match (?-mix:^https?:\/\/.+$).

[marcwrobel]

Merging this as:

• the build pass and the replacement solution has been extensively tested,
• it solves validation issues on PRs (for instance on this one the validation check is not executed),
• it improves the validation process overall,
• I would like to verify the new check-links workflow works as expected,
• it would be interesting to see which links are now invalid thanks to check-links (did not check all links for some time),
• the endoflife.date team looks pretty busy these days and I do not hope a review in the coming days.

@captn3m0 FYI

[marcwrobel]

Following merge:

• Deploy preview still deployed : Bump actions/checkout from 2 to 3 #3427
• Auto-merge release update still working as expected: Bump _data/release-data from 7ff53f7 to 8634916 #3428

[marcwrobel]

Link validation running on https://github.com/endoflife-date/endoflife.date/actions/runs/5911191568.