github docs return 403

[ustiugov]

Please check the logs here

[jtracey93]

Seeing the same here: https://github.com/Azure/ALZ-Bicep/runs/6162500034?check_suite_focus=true#step:4:61

[rfay]

I'm also seeing this. I imagine it's some protection added to the server side.

[mboersma]

Same here, also if I try to hit docs.github.com with curl (although of course my browser on the same host works fine):

% curl -I https://docs.github.com/en/authentication/managing-commit-signature-verification/adding-a-new-gpg-key-to-your-github-account
HTTP/2 403 

[invidian]

Simple bisecting unveils adding --compressed makes it work:

$ curl -I https://docs.github.com/en/authentication/managing-commit-signature-verification/adding-a-new-gpg-key-to-your-github-account
HTTP/2 403
x-azure-ref: 0AHdxYgAAAAAgXfIJ/0DbQKFiAWjsl4AlRlJBMjMxMDUwNDE4MDA5ADU5NmQ3OGEyLWNhNWYtNDc5ZC1iY2RjLTA4MzU4MzMxNzRiMg==
accept-ranges: bytes
date: Tue, 03 May 2022 18:40:00 GMT
via: 1.1 varnish
x-served-by: cache-hhn4027-HHN
x-cache: MISS
x-cache-hits: 0
x-timer: S1651603201.602146,VS0,VE19
strict-transport-security: max-age=31557600

$ curl -I https://docs.github.com/en/authentication/managing-commit-signature-verification/adding-a-new-gpg-key-to-your-github-account --compressed
HTTP/2 200
cache-control: private, no-store
content-type: text/html; charset=utf-8
etag: "27c55-cbKdgKYi/E8sC85PlPGaN3K2Lvc"
set-cookie: _csrf=3DJJ1KK_R3E8cuBGvTJE2seG; Path=/; HttpOnly; Secure; SameSite=Lax
access-control-allow-origin: *
content-security-policy: default-src 'none';prefetch-src 'self';connect-src 'self';font-src 'self' data: githubdocs.azureedge.net;img-src 'self' data: github.githubassets.com githubdocs.azureedge.net placehold.it *.githubusercontent.com github.com;object-src 'self';script-src 'self';frame-src https://graphql.github.com/ https://www.youtube-nocookie.com;style-src 'self' 'unsafe-inline';child-src 'self'
x-dns-prefetch-control: off
expect-ct: max-age=0
x-frame-options: SAMEORIGIN
x-download-options: noopen
x-content-type-options: nosniff
x-permitted-cross-domain-policies: none
referrer-policy: strict-origin-when-cross-origin
x-xss-protection: 0
x-powered-by: Next.js
x-azure-ref: 0BHdxYgAAAACVuMTMpp4JSaS4AevJbpy6RlJBMjMxMDUwNDE4MDA5ADU5NmQ3OGEyLWNhNWYtNDc5ZC1iY2RjLTA4MzU4MzMxNzRiMg==
accept-ranges: bytes
date: Tue, 03 May 2022 18:40:04 GMT
via: 1.1 varnish
x-served-by: cache-hhn4058-HHN
x-cache: CONFIG_NOCACHE, MISS
x-cache-hits: 0
x-timer: S1651603204.031860,VS0,VE167
vary: Accept-Encoding
strict-transport-security: max-age=31557600
content-length: 162901

[mboersma]

The workaround mentioned in this issue seems to fix things: tcort/markdown-link-check#201

[gaurav-nelson]

Thank you @mboersma

As a workaround, you can add the following to the config file:

  "httpHeaders": [
    {
      "urls": ["https://docs.github.com/"],
      "headers": {
        "Accept-Encoding": "zstd, br, gzip, deflate"
      }
    }
  ]

as seen here: https://github.com/openmrs/openmrs-esm-core/pull/412/files

Closing this issue as it relates to the underlying library PS: tcort/markdown-link-check#201