Automatically PR to update poetry lockfile

[sneakers-the-rat]

Following: #2002

See #2002 (comment) which links to some of the conversation and prior issues (that i'll also link at the bottom for the sake of recordkeeping)

This action runs an action once a week (or on demand) to create a pull request updating the lockfile.

The body of the PR will look something like this (eg. for the current state of the repo):

Update poetry dependencies:

  •  rpds-py (0.17.1 -> 0.18.0)
  •  typing-extensions (4.9.0 -> 4.10.0)
  •  packaging (23.2 -> 24.0)
  •  pyparsing (3.1.1 -> 3.1.2)
  •  referencing (0.33.0 -> 0.34.0)
  •  urllib3 (2.2.0 -> 2.2.1)
  •  pydantic (1.10.14 -> 2.6.4)
  •  traitlets (5.14.1 -> 5.14.2)
  •  curies (0.7.7 -> 0.7.8)
  •  jupyter-core (5.7.1 -> 5.7.2)
  •  python-dateutil (2.8.2 -> 2.9.0.post0)
  •  types-python-dateutil (2.8.19.20240106 -> 2.9.0.20240316)
  •  filelock (3.13.1 -> 3.13.3)
  •  jupyter-client (8.6.0 -> 8.6.1)
  •  nbformat (5.9.2 -> 5.10.3)
  •  prefixmaps (0.1.7 -> 0.2.2)
  •  cachetools (5.3.2 -> 5.3.3)
  •  comm (0.2.1 -> 0.2.2)
  •  debugpy (1.8.0 -> 1.8.1)
  •  linkml-runtime (1.7.0 -> 1.7.4)
  •  nbclient (0.9.0 -> 0.10.0)
  •  virtualenv (20.25.0 -> 20.25.1)
  •  graphviz (0.20.1 -> 0.20.3)
  •  ipykernel (6.29.2 -> 6.29.3)
  •  nbconvert (7.16.0 -> 7.16.3)
  •  sqlalchemy (2.0.25 -> 2.0.29)
  •  tox (4.12.1 -> 4.14.2)

this forces our development environment to be within a week of what ppl would experience if they were to install linkml with pip (which will install the most recent version of all packages matching the version spec in pyproject.toml). Note how it is updating pydantic here which we have kept locked at 1.10.14 by sheer force of not updating the lockfile (see 1925 and 1957).

The create-pull-request action will keep updating the same pull request if it is not merged by the time the next run happens, so we won't stack up a bunch of lockfile PRs. We can set these up to automatically merge if all tests pass if we try this out for awhile and decide that would be a good idea - that could be something we set up as an additional step after the main tests, since it looks like the github branch protections-based automerge needs to be explicitly enabled for each PR, which makes sense. I figured we might as well get the manual version going first before we try and automate that since inevitably there will be some kinks to work out.

I'm also not sure if i need to do more to configure the token, but i suppose we will see on first run

also sorry if i keep making new labels and that's not wanted - i set the PR to use the poetry label so that we can quickly see all the PRs made by this action, distinct from the more general devops label which includes all devops stuff. i can change that and delete the label if we don't like that.

Prior related issues:

• Outdated lockfile hides test failures #1641
• We need to test before release. linkml-runtime release v1.6.1 is broken, and it's impacting the rest of the supply chain (e.g. bmt toolkit) #1749
• rdflib v7 introduces breaking changes #1567 (and i think this might partially close that)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.69%. Comparing base (9df3d3b) to head (043ceeb).
Report is 2 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2023   +/-   ##
=======================================
  Coverage   80.69%   80.69%           
=======================================
  Files         104      104           
  Lines       11622    11622           
  Branches     2910     2910           
=======================================
  Hits         9378     9378           
  Misses       1701     1701           
  Partials      543      543           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sneakers-the-rat]

perhaps ironically, it looks like the tests are failing because pyshacl isn't present, presumably because the lockfile is old and thus the cache hasn't been invalidated. confirmed pyshacl is installed from a fresh clone with -E tests

[hrshdhgd]

Awesome! thanks @sneakers-the-rat !

[sneakers-the-rat]

we'll see if it works! wasn't able to test it myself because yno github actions hard to test except for running them as actions.

[pkalita-lbl]

I'm also not sure if i need to do more to configure the token

I expect we will. Since this is using the default GITHUB_TOKEN I expect we'll see a PR get created but test runs won't be triggered by the opening of that PR. That's just based on my reading of https://github.com/peter-evans/create-pull-request/blob/main/docs/concepts-guidelines.md#triggering-further-workflow-runs. I haven't dug into the suggested workarounds yet to understand which might be the best for us.

[sneakers-the-rat]

@pkalita-lbl lets fix it in v2 ya? I can never get CI stuff right on the first time lol
#2028

[hrshdhgd]

Aah ...good point @pkalita-lbl , the solution to that is:

uses: peter-evans/create-pull-request@v6
       with:
             token: TOKEN

So TOKEN needs to be created by a general account. I say this because if it is made from a personal GitHub account, every PR created by this workflow will be attached to the user and it will appear like that particular user started the PR which is not true. Passing the token ensures triggering of the CI stuff.

[pkalita-lbl]

lets fix it in v2 ya?

Yes, of course. My apologies I didn't see that new PR when I had commented here.

every PR created by this workflow will be attached to the user and it will appear like that particular user started the PR

Moreover that person won't be able to review or merge that PR.

created by a general account

I haven't heard of a general account before. Are there GitHub docs you can share on that?

[hrshdhgd]

Well, there isn't documentation on this (generated by us at least). Having said that @balhoff has ontobot that is a general bot user that makes PRs. Could you enlighten us on how you made this bot @balhoff ? Maybe linkml could have a linkml-bot?

[sneakers-the-rat]

Ya ya lets make a bot user :) we'll need it for linkml-runtime eventually too, no?

[balhoff]

@hrshdhgd ontobot is just a regular user I made. I'm not sure if there is a way to indicate that it's a bot.

[hrshdhgd]

@hrshdhgd ontobot is just a regular user I made. I'm not sure if there is a way to indicate that it's a bot.

Aah! That makes sense.