Test Workflow Action

Sample Docker-based Github action to automatically test a PW workflow using the PW client. The action defines the following steps:

1. Creates a cluster on the PW platform
2. Configures the cluster
3. Starts the PW resources required by the workflow
4. Submits the workflow
5. Stops the PW resources started in (3)
6. Deletes the cluster created

The inputs to the action are defined in the action.yml file here and are general so they apply to almost any PW workflow. To add this action to a PW workflow (stored in a separate repository since this repository stores only the action), there are at least two steps that are done in the workflow repository:

1. The PW client requires the API key of the account for authentication. Store this key as a Github secret in the repository with your PW workflow (not this repository). Navigate to your workflow on GitHub.com and then click on Settings > Secrets > Actions > New Repository Secret. This will only be possible if you are the owner of the workflow repository or have been granted permissions to edit/view repository secrets in the workflow repository, otherwise the buttons will be hidden.
2. Tell GitHub that this action needs to be triggered whenever something (that you specify) happens in the workflow repository by adding this action to .github/workflows/main.yaml in the workflow repository. For example, the code snippet below adds this action the Github repository of a PW workflow such that the worflow is tested with every new push:

on: [push]

jobs:
  test-pw-workflow:
    runs-on: ubuntu-latest
    name: test-pw-workflow-cloud
    steps:
      - name: run-workflow-cloud
        id: run-cloud
        uses: parallelworks/test-workflow-action@v5
        with:
          pw-user-host: 'cloud.parallel.works'
          pw-api-key: ${{ secrets.PW_API_KEY }}
          pw-user: 'jlin'
          resource-pool-name: 'gcpslurmv2'
          resource-pool-tpe: 'gclusterv2'
          workflow-name: 'test'
          workflow-parameters: '{"command": "hostname", "resource_1": {"type": "computeResource", "id": "6489decae2199bee7eaf7a7f"}, "startCmd": "001_single_resource_command/main.sh"}'

The last five lines of this example of .github/workflows/main.yaml are the 6 essential inputs to running any PW workflow via the PW API. These inputs will vary depending on the host (e.g. cloud.parallel.works), the user's credentials (API key and username), the resource name and type to be created, and the workflow itself (workflow-name and workflow-parameters); as such all 6 of these lines need to be specific to a particular workflow and are defined in the workflow repository and not in the "action repository" (here).

Also note that the resource ID will be replaced by the created resource ID, and startCmd will be replaced by the one specified in workflow.xml.

Notes:

1. The workflow-parameters for your workflow can be downloaded from the input form in PW as shown in the screenshot below. Using the platform-generated version of the workflow-parameters is a great starting point, especially when attempting to API-launch workflows with many parameters. To compare a .github/workflow/main.yml file's workflow-parameters to the ones generated by the platform, you can use check_gh_api_launch.sh </path/to/your/main.yml> in this repository

2. GitHub, by default does not allow users to use their PAT to add .github/workflows/main.yaml. Either add this permission to your PAT via your account's Settings > Developer Settings > Personal Access Tokens, selecting your PAT, and checking the workflow box or use an SSH key. An easy alternative is to use the GitHub GUI to add actions by clicking on the Actions tab for your repository and then clicking on the set up a workflow yourself -> link. This issue is purely permission to add an action to a repository - once that permission is added, a standard PAT (without any special GitHub workflow permission) can still make pushes to the repository, and if the action is already set up to run on a push, then those actions will be executed.

3. If instead of a push the GH action in the workflow repository is launched by a GH release, be careful to specify a specific stage in the release process (creation, editing, publishing) - otherwise a general on: [release] will actually be interpreted at three separate events, each event lauching a different workflow run. Explicitly, to select the publishing of a release (instead of creation or editing), the following header snippet can be used:

on:
  release:
    type: [publish]
    
jobs:
  ...

4. Setting the cluster ID will be dependent on the workflow parameter JSON. This can (and probably will) be changed in run_workflow.py for future benchmark tests (although there could be a better way to do this, like setting the cluster ID as a fixed variable in the workflow JSON file for each workflow, much like startCmd). resource.json can also be changed to reflect the cluster configuration needed for specific benchmark tests, but this creates a basic and cheap Google cluster.