This repo needs tests

[brian-rose]

In ProjectPythia/cookbook-template#61 I'm working on refactoring how we handle all the Cookbook actions to reduce boilerplate in each Cookbook repo and centralize more of the workflow code here. This is great from a maintenance perspective. But it creates a new problem:

Changes to this repo (e.g. updating versions of its dependencies) won't directly trigger any useful tests. We currently need to wait until one or more of the Cookbook repos fail before knowing there's a problem.

One approach might be to trigger a test build of the Cookbook Template repo on every PR to this repo.

[brian-rose]

I think it may be possible to write a workflow triggered by PRs to this repo that

1. checks out the Template repo
2. runs a build on that repo, using the latest versions of the build-book.yaml etc that are in the PR.
3. Displays a preview of that build, again using the latest versions of the yaml files.

That should be an effective way to make sure changes in each PR don't break anything. Displaying the preview would require turning on GitHub Pages for this repo.

Workflow code to check out a different repo should look like this:

- name: Check out my other repo
        uses: actions/checkout@v3
        with:
          repository: orgname/reponame

[brian-rose]

@ProjectPythiaCookbooks/infrastructure I think this is a high priority since we're relying on the infrastructure in this repo, so I took the liberty of labelling it as such. I need some help with implementing a solution.

[brian-rose]

@ProjectPythia/infrastructure I want to draw attention to this as a high-priority issue.

With ProjectPythia/pythia-foundations#367 and ProjectPythia/projectpythia.github.io#310, almost all of our content (including Foundations and the portal site) is now depending on the workflows in this repo. But this repo doesn't have its own tests, so failures due to upstream changes would only be caught indirectly via their effects on our nightly builds in other repos.

Designing proper tests for this repo is non-trivial, as I outlined above. It would be great to get someone assigned to look into this.

[dcamron]

I'm going to start on this. I'll start with the goal for now of "check if cookbook-template builds successfully provided changes to these actions". Let me know if anything else comes up as important to check, and I might ask some questions here as I dig in.

[brian-rose]

I'm going to start on this. I'll start with the goal for now of "check if cookbook-template builds successfully provided changes to these actions". Let me know if anything else comes up as important to check, and I might ask some questions here as I dig in.

I think that's the right approach.

The important detail to keep an eye on is to make sure the correct version of the actions are called. The workflow files in the template repo point specifically to the main branch of this repo, so if you just run them without modification, they won't be testing any new changes in the PR on this branch.

[dcamron]

ProjectPythia/cookbook-template#100 (comment) a note about confirming our testing methodology across actions. This may deserve its own issue.

[clyne]

Highest priority item for Drew at the moment.