BUG: our contributor build is currently broken (this is likely a pyosmeta bug)

[lwasser]

We have a but in our contributor build here that is causing it to fail.

https://github.com/pyOpenSci/pyopensci.github.io/actions/runs/13272538893/job/37055135801

[pllim]

I don't wanna go too deep but I think this is the path I dug up

pyopensci.github.io/.github/workflows/update-contribs-reviews.yml

Line 30 in 73c1b19

update-reviews

https://github.com/pyOpenSci/pyosMeta/blob/0baa1542d436320bfce6299d59c7263aa4a5a8d1/src/pyosmeta/parse_issues.py#L436

Processing review EarthPy: Software Submission for Review
    reviews[pkg_name].gh_meta = pkg_meta[pkg_name]
  File "pydantic/main.py", line 922, in __setattr__
    self.__pydantic_validator__.validate_assignment(self, name, value)
pydantic_core._pydantic_core.ValidationError: 1 validation error for ReviewModel
gh_meta.description
  Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
    For further information visit https://errors.pydantic.dev/2.10/v/string_type

Without actually running the stuff locally, I cannot tell if it is trying to parse

pyopensci.github.io/_data/packages.yml

Lines 2042 to 2043 in 73c1b19

	description: A package built to support working with spatial data using open
	source python

, pyOpenSci/software-submission#3 , or a file from https://github.com/earthlab/earthpy

[pllim]

Looks like it has been a problem for 2 weeks: https://github.com/pyOpenSci/pyopensci.github.io/actions/workflows/update-contribs-reviews.yml?query=branch%3Amain

[lwasser]

Based on that field gh_meta.description the gh_meta object is what we use to grab metadata for each package! You are correct; it's something associated with validating that field!! Thank you for looking into it. The actual description l ooks ok on the package

https://github.com/earthlab/earthpy

We should actually also sunset Earthpy as it's no longer maintained.
@banesullivan when you work on this let's also archive earthpy! but fix the bug first as it could come up again!

[pllim]

The pydantic error message could have been more verbose but that is neither here nor there... 😅

[lwasser]

🤣 posmeta has a few features like this
This is a new one. The other one that is my favorite is if your token isn't setup properly it just quietly runs and does nothing.
🙃

I'm sure there are other gems in the code too.

[banesullivan]

I've landed a hotfix for this in pyOpenSci/pyosMeta#257, below are my notes from this incident

Sequence of Events

• Moritz reports issue with package not showing up on website
• Leah responds assuming a build just needs to be triggered
• Moritz attempts to run update and notices several warnings
• Leah provides reasurance warnings are normal
• Leah then notices that the thztools author deleted one of the YAML headings so it couldn’t be parsed (BUG TO ADDRESS?). Leah manually fixed the review and reran.
• Moritz suggests a daily cron job to help narrow down these errors sooner

Issue opened here for CI failure. Problem has been around for ~2 weeks at this point

Processing review EarthPy: Software Submission for Review
    reviews[pkg_name].gh_meta = pkg_meta[pkg_name]
  File "pydantic/main.py", line 922, in __setattr__
    self.__pydantic_validator__.validate_assignment(self, name, value)
pydantic_core._pydantic_core.ValidationError: 1 validation error for ReviewModel
gh_meta.description
  Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
    For further information visit https://errors.pydantic.dev/2.10/v/string_type

• Problem package is thought to be EarthPy
• Leah mentions we should archive EarthPy as well (comment)
• Leah also mentions that pyosmeta will silently fail if the github token is missing
• Bane starts investigating
• The failure is coming from the get_gh_metrics step of the src/pyosmeta/cli/process_reviews.py script
• EarthPy is not the problem, it just happens to be the last printed in the previous parse_issues step.
• Adding logs to the get_gh_metrics funstion show the problem package to be astrodata
• The issue is in process_repo_meta returning None for the description. If we look at astrodata's GitHub page there indeed is no description. This is something our parsers should handle

Immediate To Do's

• Fix validation issue and see if we can make the validation errors more verbose/meaningful
  • add a print out/log to indicate each step of the script
  • make sure that the process_repo_meta utility returns values that conform to GhMeta and that any null values are supported: hotfix handled in [hotfix] handle packages that lack a description on GitHub pyosMeta#257
• Archive EarthPy
• Make sure pyosmeta errors when token is missing per Leah's comment. I looked into this and of the few ways I misconfigured my token, the script failed with an appropriate error message regarding the token

Follow Up To Do's

We should probably open seperate issues for some of these

• We should use tqdm for progress monitoring since we have a lot of print outs
• A good number of warnings are thrown, let's define a custom warning type and make sure its clear what's a problem and what's okay.
• Make sure a missing "community partnerships" section is tested
• Can we set up a Slack Alert that pings us any time the build is failing like this?
• Address what went wrong with thztools and see if we can gracefully handle it - The One-Line Description of Package: field label was removed and it hust had the description

[pllim]

Thanks! Does something else still need to run after the fix because I still don't see the new package listed under Astropy page.

[pllim]

P.s. Astropy tried the Slack thingy years ago but it didn't work. Slack has GitHub app but it's for issues, not workflows. Might be something new now so maybe worth checking their app selections again.

[banesullivan]

For posterity, we ended up using https://github.com/marketplace/actions/slack-notify in #584 to alert us of failures moving forward.

I think everything here has been addressed in regards to the original failure/bug with follow on work captured in pyOpenSci/pyosMeta#270. Closing this issue, but let me know if we feel anything was left unaddressed