Check links fails too often CI due to rate limitting

[krassowski]

Description

On CI check links fails too often, here are some common rate-limited endpoints:

_ /home/runner/work/jupyterlab/jupyterlab/docs/source/developer/accessibility.rst: https://learn.microsoft.com/en-us/microsoft-edge/devtools-guide-chromium/accessibility/emulate-vision-deficiencies#open-the-rendering-tool _
https://learn.microsoft.com/en-us/microsoft-edge/devtools-guide-chromium/accessibility/emulate-vision-deficiencies#open-the-rendering-tool: 429: Too Many Requests

_ /home/runner/work/jupyterlab/jupyterlab/docs/source/extension/extension_tutorial.rst: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development _
https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development: 429: Too Many Requests

[jtpio]

Normally the maintainer-tools/base-setup action should cache check links result to avoid such issues. Maybe this is not working as expected.

[krassowski]

Another one:

_ /home/runner/work/jupyterlab/jupyterlab/docs/source/extension/ui_components.rst: https://stackoverflow.com/a/48198011/425458 _
https://stackoverflow.com/a/48198011/425458: 403: Forbidden

[krassowski]

Recent ones https://github.com/jupyterlab/jupyterlab/actions/runs/8124321561/job/22246235399?pr=15898

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/rursprung>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/yumyumqing>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/damiend97>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/zrottman>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/kamalika0363>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/jmk89>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/yczhangsjtu>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/joaopalmeiro>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/dlqqq>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/vthinkxie>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/kulsoomzahra>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/kulsoomzahra>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/matthewturk>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/ajbozarth>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/athornton>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/hsuanxyz>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/hsuanxyz>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/thomasaarholt>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/aditya211935>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/dlqqq>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/dlqqq>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/dlqqq>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/dlqqq>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/hsuanxyz>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/joaopalmeiro>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/dlqqq>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/dlqqq>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/joaopalmeiro>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/jmk89>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/vthinkxie>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/hsuanxyz>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/hsuanxyz>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/thomasaarholt>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/ajbozarth>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/ajbozarth>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/ajbozarth>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/ajbozarth>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/ajbozarth>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/ajbozarth>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/ajbozarth>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/ajbozarth>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/ajbozarth>
FAILED CHANGELOG.md::/home/runner/work/jupyterlab/jupyterlab/CHANGELOG.md <a href=https://github.com/ajbozarth>
43 failed, 2732 deselected, 1 warning in 3.00s

[krassowski]

I guess jupyterlab/maintainer-tools#205 would help here.

[krassowski]

Normally the maintainer-tools/base-setup action should cache check links result to avoid such issues. Maybe this is not working as expected.

I see that there is links_expire indeed which is supposed to last one week. In the failing job linked above I see:

Run actions/cache@v3
Cache Size: ~19 MB (19447820 B)
/usr/bin/tar -xf /home/runner/work/_temp/a22c540e-b2fb-4f33-9d5c-04468c0f6fb0/cache.tzst -P -C /home/runner/work/jupyterlab/jupyterlab --use-compress-program unzstd
Cache restored successfully
Cache restored from key: Linux-Linux Tests-check_links-pip-3.11-e26b230aa3718845779a662c894fa21240a573a90a838156b834f084182d250c
Run actions/setup-node@v3
Found in cache @ /opt/hostedtoolcache/node/18.19.1/x64
Environment details
Run CACHE_DIR=$(yarn config get cacheFolder)
Run actions/cache@v3
Cache Size: ~142 MB (148833560 B)
/usr/bin/tar -xf /home/runner/work/_temp/c9cadcf4-2c20-49ff-941b-ef3333a5832c/cache.tzst -P -C /home/runner/work/jupyterlab/jupyterlab --use-compress-program unzstd
Cache restored successfully
Cache restored from key: Linux-Linux Tests-check_links-yarn-18.x-7df5af20877d5f4f03259bdfb2db6c232e4a0596c76c75b20a1e5487722bc6a2
Run actions/cache@v3
Cache not found for input keys: Linux-Linux Tests-check_links-conda--18e9f0acd2b71593cb2fd4b3bcdad4856039b105639de96155d51a9650750743, Linux-Linux Tests-check_links-conda-

I am not sure how to interpret seeing both "Cache restored successfully" and "Cache not found for input keys".

Another option at least for that one specific issue of changelog is to ignore usernames and organisations by adding "https://github.com/[^/]+ pattern to ignore list.

@blink1073 do you have any advice here?

[blink1073]

I don't have the bandwidth to debug, but we are in fact ignoring GitHub links already https://github.com/jupyterlab/maintainer-tools/blob/5de3cd5d1953dce710fff2cf16a576ea209b9d3f/.github/actions/check-links/check_links.py#L34

[krassowski]

Yes, just not this specific pattern. I am leaning towards just adding that pattern for now.

[krassowski]

I opened up jupyterlab/maintainer-tools#221 because it it feels like the false positive ratio for user profiles is too high even if sometimes it does catch an org moving or a typo. Possibly best of both words would be ignoring user profile links in CHANGLOG.md only but this seems a bit more involved.

[linlol]

I guess either one would help resolve the current issue. If CHANGELOG.md is auto-generated, maybe we can just ignore the whole file, otherwise maybe ignore user-profile pattern is better?

I opened up jupyterlab/maintainer-tools#221 because it it feels like the false positive ratio for user profiles is too high even if sometimes it does catch an org moving or a typo. Possibly best of both words would be ignoring user profile links in CHANGLOG.md only but this seems a bit more involved.

[krassowski]

Parts of changelog are auto-generated, but parts such as user-facing changelog are manually curated and do include links which people want to click on.