Use dois for all links

[mgrover1]

It looks like our CI is failing due to broken links.

All of the references to AGU journals claim there are broken links. I think we should use the permanent DOIs as the links here.

One other cause might be the new AGU terms of use, which mentions:

1. Users may browse all Tables of Content for all Electronic Products on Wiley Online Library, regardless of what Licensed Electronic Products the Customer subscribes to, and search the entire database of Tables of Content and abstracts. For most publications, Users may browse the abstracts as well.

2. Customers and their Authorized Users will have access to the full text of the Licensed Electronic Products covered by the applicable License. Customers and their Authorized Users may download, search, retrieve, display and view, copy and save to a secure network or other electronic storage media and store or print out single copies of individual articles or items for their own personal use, scholarly, educational or scientific research or internal business use. Customers and Authorized Users may also transmit such material to a third-party colleague in hard copy or electronically for personal use or scholarly, educational, or scientific research or professional use but in no case for re-sale, systematic distribution, e.g. posting on a listserv, network (including scientific social networks) or automated delivery, or for any other use (including distribution through social networking websites and scholarly collaboration networks, except for those that have agreed to Wiley’s Article Sharing Policy found here: https://authorservices.wiley.com/author-resources/Journal-Authors/Promotion/article-sharing-policy.html and solely in accordance therewith). In addition, Authorized Users have the right to use, with appropriate credit, figures, tables and brief excerpts from individual articles in the Licensed Electronic Product(s) in their own scientific, scholarly and educational works. Please note that these rights do not extend to the use of material, images or figures that are separately listed as the copyright of a third party.

3. All Users have the option to create a My Profile Page which will allow them to create links to Electronic Product(s), articles of interest and search criteria which may be reused by them and manage their custom e-mail alerting services. In order to do so, Users must register. During the registration process, Users will select and register a user name and password which they must keep confidential and not disclose to or share with anyone else.

I am curious if this somehow blocks anonymous checks of the sites...

[github-actions]

👋 Thanks for opening this PR! The Cookbook will be automatically built with GitHub Actions. To see the status of your deployment, click below.
🔍 Git commit SHA: 70e8f36
✅ Deployment Preview URL: https://ProjectPythia.github.io/cmip6-cookbook/_preview/46

[mgrover1]

@brian-rose - do you think skipping DOI patterns in the link checking could be at the cookbook action level (instead of each cookbook)?

[brian-rose]

@brian-rose - do you think skipping DOI patterns in the link checking could be at the cookbook action level (instead of each cookbook)?

Yes I think we can do that. It's a bit annoying that we need to, because there's nothing wrong with the DOI links but somehow the redirect to the AGU journal sites is causing problems for the link-checker. But since the whole point of DOI links is that they are persistent, there's also no good reason to consume CI resources by checking them continually.

The place to implement the DOI pattern exclusion would be in the shared link checker workflow here. We'll have to do a bit of digging to figure out how to do it.

[brian-rose]

In general it's good practice to use DOI links rather than links to publisher pages for any published papers, since they are persistent. It would be great to get this sage advice somewhere in our template or contributors guide.

[mgrover1]

Yeah - we can add that to the docs. I agree that it is frustrating in this case, especially since it seems to only be AGU articles causing the issues...

[mgrover1]

Should we go ahead and merge this @brian-rose , and we can resolve the linkcheck upstream in the cookbook-actions repo?

[mgrover1]

I tried fixing it using the config file... we shall see

[brian-rose]

I tried fixing it using the config file... we shall see

Well that didn't work. I say remove the config file change here, let's merge the DOI change, and we'll address the link checker issue elsewhere.

[mgrover1]

Well that didn't work. I say remove the config file change here, let's merge the DOI change, and we'll address the link checker issue elsewhere.

Done! @brian-rose

[brian-rose]

Now I'm confused because the link-checker is still failing but seemingly for other reasons. We're getting

Could not import extension sphinx_book_theme (exception: cannot import name '_get_theme_options' from 'pydata_sphinx_theme' (/usr/share/miniconda3/envs/test/lib/python3.11/site-packages/pydata_sphinx_theme/__init__.py))

[brian-rose]

I just found the same failure by manually triggering the nightly-build on the template repo: https://github.com/ProjectPythia/cookbook-template/actions/runs/4568476927/jobs/8063584456

Something new has broken upstream 😭

[mgrover1]

I am not sure here...

[brian-rose]

This was just released 5 hours ago... https://github.com/pydata/pydata-sphinx-theme/releases/tag/v0.13.3

[mgrover1]

Oh no... it looks like there was a big refactor...

[brian-rose]

Yup, I just confirmed that basic jupyter-book functionality is broken:

mamba create --name testjbook -c conda-forge jupyter-book
conda activate testjbook
jupyter-book create mynewbook/
jupyter-book build mynewbook/

triggers the same error on my Mac laptop.

[brian-rose]

Now to figure out the best place to report this...

[brian-rose]

Already reported here:
executablebooks/sphinx-book-theme#711

[brian-rose]

The upstream problem got resolved quickly with the release of sphinx-book-theme 1.0.1, yay!

We're now back to the expected link-check failures:

(notebooks/example-workflows/xesmf-ohu: line 40004) broken    https://doi.org/10.1029/2019AV000130 - 403 Client Error: Forbidden for url: https://onlinelibrary.wiley.com/doi/abs/10.1029/2019AV000130
[64](https://github.com/ProjectPythia/cmip6-cookbook/actions/runs/4576219219/jobs/8080055622?pr=46#step:7:65)
(notebooks/example-workflows/ecs-cmip6: line 40006) broken    https://doi.org/10.1029/2003GL018747 - 403 Client Error: Forbidden for url: https://onlinelibrary.wiley.com/resolve/doi?DOI=10.1029/2003GL018747
[65](https://github.com/ProjectPythia/cmip6-cookbook/actions/runs/4576219219/jobs/8080055622?pr=46#step:7:66)
(notebooks/example-workflows/ecs-cmip6: line 40009) broken    https://doi.org/10.1029/2018MS001400 - 403 Client Error: Forbidden for url: https://onlinelibrary.wiley.com/doi/10.1029/2018MS001400

[brian-rose]

As discussed above, we will fix the link check failures upstream by excluding doi.org patterns from the check for all cookbooks.