Certificates: nersc.gov, ornl.gov

[ax3l]

Hi,

Using this action I have problems verifying nersc.gov and ornl.gov certificates in a standard Ubuntu-20.04 GH action instance:

HTTPSConnectionPool(host='docs-dev.nersc.gov', port=443): Max retries exceeded with url: /cgpu/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)')))
https://docs-dev.nersc.gov/cgpu/

HTTPSConnectionPool(host='www.olcf.ornl.gov', port=443): Max retries exceeded with url: /summit/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)')))
https://www.olcf.ornl.gov/summit/

Googling the interwebs, I think that we can fix this by also pre-installing certifi into the actions:

• https://stackoverflow.com/a/28725127/2719194
• https://stackoverflow.com/questions/50236117/scraping-ssl-certificate-verify-failed-error-for-http-en-wikipedia-org

[vsoch]

hey @ax3l that's not an issue with the action, there is an SSL certificate error. You need to add those patterns to the whitelist patterns, which are defined in both the linters.yaml (with urlchecker) and the nightly task to clean jobs. Thanks!

[vsoch]

If you like I'll make a branch for you here to test that installs that dependency. Give me a sec.

[vsoch]

Here you go! https://github.com/urlstechie/urlchecker-action/tree/add/certifi Feel free to PR to that branch if you want to fuss around further. Thanks!

[ax3l]

Thanks a lot!
I quickly built this out via

docker build -t url .

and run it via

docker run -t url -v "$(pwd)":/github/workspace -e INPUT_FILE_TYPES=".py,.cpp,.H,.rst,.md" --workdir /github/workspace --rm

But it does not seem to find my files in my PWD. Any other hints I need to pass into the container?

[ax3l]

Oh..., I can try this branch in an action via uses: urlstechie/urlchecker-action@add/certifi 💡

[ax3l]

I tried a couple further updates - down to even doing:

FROM quay.io/urlstechie/urlchecker:0.0.22
RUN apt-get update && \
    /bin/bash -c "install_packages ca-certificates && \
    source activate urlchecker && \
    which python && \
    which pip && \
    python -m pip install --upgrade pip && \
    python -m pip install --upgrade certifi requests && \
    python -c 'import certifi; print(certifi.where())'"
# prints: /opt/conda/lib/python3.8/site-packages/certifi/cacert.pem from certifi-2021.5.30

#ENV REQUESTS_CA_BUNDLE /etc/ssl/certs/ca-certificates.crt
COPY entrypoint.sh /entrypoint.sh
WORKDIR /github/workspace
ENTRYPOINT ["/bin/bash", "/entrypoint.sh"]

I think one problem might be that the base image bases on Debian 9 "stretch": https://www.debian.org/releases/stretch/

Debian 9 has been superseded by Debian 10 ("buster"). Security updates have been discontinued as of July 6th, 2020. 

Shall we just update to Debian 11 ("Bullseye")?
https://github.com/urlstechie/urlchecker-python/blob/08fc1bb93beaf1263701dc7c1e609a399bc7a32d/Dockerfile#L1

[vsoch]

I could try, we would want to do that anyway. Let me open a PR on that other repo and see if it still works. I might as well at certifi to that container base too.

[ax3l]

Got it :) urlstechie/urlchecker-python#58

[vsoch]

see #80

[vsoch]

@ax3l I noticed from our discussion day that GitHub changed the UI for showing icons alongside comments - it's in a little nested bubble now instead of anchored to the bottom!

The weird thing is that I still have tabs open with the old format!

[ax3l]

Yes, same for me - changed mid day, I thought I broke my icon sets or something 😅

[ax3l]

Root problem found:

• https://www.ssllabs.com/ssltest/analyze.html?d=www.olcf.ornl.gov&latest
• https://www.ssllabs.com/ssltest/analyze.html?d=docs%2ddev.nersc.gov&latest

have indeed chain issues.

[vsoch]

Ah, there we go!

It was still worth the work because we updated the container base! I'll leave this issue open so we can discuss again / re-test when the time comes, and I'll hold off releasing any 0.0.23 on pypi (since it's technically the same).

[ax3l]

Alrighty!
I actually think you can merge & release this updated image if you like.

I'll keep my linked WarpX PR open and will report back if the issue stays once the tickets are resolved. No need to wait for those.

NERSC: INC0172063
OLCF: OLCFHELP-3254

[vsoch]

sounds good! I'll merge the PR here so the action uses the updated image, and we can follow up after. Thanks for your work today!

[ax3l]

Thank you as well! Always a pleasure! 🎉
Have a nice evening!

[ax3l]

OLCF has fixed their certificate issue today.

[vsoch]

hooray!

[ax3l]

NERSC has also fixed their missing intermediate certificate download for docs-dev.nersc.gov