Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .eslintrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
"SharedArrayBuffer": "readonly"
},
"parserOptions": {
"ecmaVersion": 2018
"ecmaVersion": 2020
},
"rules": {
}
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/check-dist.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,11 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: Set Node.js 16.x
- name: Setup Node.js
uses: actions/setup-node@v4.0.1
with:
node-version: 16.x
node-version-file: package.json
cache: npm

- name: Install dependencies
run: npm ci
Expand Down
58 changes: 58 additions & 0 deletions .github/workflows/examples/example.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: Example Workflow
on:
pull_request
jobs:
discover-python-projects:
name: Discover projects
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v4

- name: Discover Projects
id: discover
uses: BrandonLWhite/find-python-projects-action@main

- name: Print Paths
run: echo ${{steps.discover.outputs.paths }}

outputs:
testable-projects: ${{ steps.discover.outputs.testable-projects }}
packageable-projects: ${{ steps.discover.outputs.packageable-projects }}

build-python-projects:
name: "Test: ${{ matrix.project.name }}"
runs-on: ubuntu-latest
needs: discover-python-projects
strategy:
matrix:
project: ${{ fromJson(needs.discover-python-projects.outputs.testable-projects) }}

defaults:
run:
shell: bash
working-directory: ${{ matrix.project.directory }}

steps:
- name: Check out code
uses: actions/checkout@v4

- name: Dump project object
run: |
echo '${{ toJson(matrix.project) }}'

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version-file: ${{ matrix.project.path }}

- name: Install project
if: matrix.project.installCommand
run: ${{ matrix.project.installCommand }}

- name: Test project
run: ${{ matrix.project.testCommand }}

- name: Package project
if: matrix.project.packageCommand
run: ${{ matrix.project.packageCommand }}
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,4 @@ jobs:
- uses: actions/checkout@v4
- uses: ./
with:
milliseconds: 1000
root-dir: ./test-fixtures/multi-project
2 changes: 1 addition & 1 deletion CODEOWNERS
Original file line number Diff line number Diff line change
@@ -1 +1 @@
* @actions/actions-runtime
# Repository CODEOWNERS
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) 2019 GitHub Actions
Copyright (c) 2024 Brandon L White

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
129 changes: 47 additions & 82 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,116 +1,81 @@
# Create a JavaScript Action
# find-python-projects-action
Github Action for dynamically discovering Python projects in a repository and making available some helpful values from `pyproject.toml`, including CI/CD commands like `test`, for each project it finds.

<p align="center">
<a href="https://github.com/actions/javascript-action/actions"><img alt="javscript-action status" src="https://github.com/actions/javascript-action/workflows/units-test/badge.svg"></a>
</p>
Python projects are identified by the presence of `pyproject.toml`.
Various pieces of information about each project are parsed from `pyproject.toml` and returned in the action outputs as JSON strings, including shell commands for additional CI type operations like `test` and `package`.
This is meant to faciliate downstream actions or workflows such as matrix builds for parallel builds of each project.

Use this template to bootstrap the creation of a JavaScript action.:rocket:
This action aims to help you eliminate (or at least reduce) the amount of customization needed in your GHA workflows by pushing your project-specific stuff into `pyproject.toml`.

This template includes tests, linting, a validation workflow, publishing, and versioning guidance.

If you are new, there's also a simpler introduction. See the [Hello World JavaScript Action](https://github.com/actions/hello-world-javascript-action)
## Inputs
- **root-dir**: Directory root for where to begin recursively searching for projects.
Python projects contained in this directory or lower will be discovered. Defaults to your repository's root directory.

## Create an action from this template

Click the `Use this Template` and provide the new repo details for your action
## Outputs
- **paths**: JSON array of found project path strings

## Code in Main
- **projects**: JSON array of all found projects ([project object](#project-object-output-shape))

Install the dependencies
- **testable-projects**: JSON array of all found projects ([project object](#project-object-output-shape)) that implement a `test` command in `pyproject.toml` (See [Project Commands](#project-commands))

```bash
npm install
```
- **packageable-projects**: JSON array of all found projects ([project object](#project-object-output-shape)) that implement a `package` command in `pyproject.toml` (See [Project Commands](#project-commands))

Run the tests :heavy_check_mark:

```bash
$ npm test
## Project object output shape
These are the fields for `project` objects in the output:

PASS ./index.test.js
✓ throws invalid number (3ms)
✓ wait 500 ms (504ms)
✓ test runs (95ms)
...
```
- **name**: The name of the project determined by `[project.name]` or `[tool.poetry.name]`

## Change action.yml
- **path**: The file path to the `pyproject.toml` file for the project.

The action.yml defines the inputs and output for your action.
- **directory**: The directory path where the `pyproject.toml` file resides.

Update the action.yml with your name, description, inputs and outputs for your action.
- **buildBackend**: Value of `[build-system.build-backend]` from `pyproject.toml`

See the [documentation](https://help.github.com/en/articles/metadata-syntax-for-github-actions)
- **pythonVersion**: Value of `[project.requires-python]` or `[tool.poetry.dependencies.python]` from `pyproject.toml`

## Change the Code
- **installCommand**: The shell command to run to create and install dependencies into the virtual environment.

Most toolkit and CI/CD operations involve async operations so the action is run in an async function.
- **testCommand**: The shell command to run to execute the tests for the project.

```javascript
const core = require('@actions/core');
...
- **packageCommand**: The shell command to run to execute packaging operations for the project.

async function run() {
try {
...
}
catch (error) {
core.setFailed(error.message);
}
}

run()
```
## Project Commands
In the absence of Python standards for expressing internal project CI/CD/Dev operations, this action tries to unify the various known ways in the wild.

See the [toolkit documentation](https://github.com/actions/toolkit/blob/master/README.md#packages) for the various packages.
This action recognizes the following typical CI related shell commands:

## Package for distribution
- `test`
- `package`

GitHub Actions will run the entry point from the action.yml. Packaging assembles the code into one file that can be checked in to Git, enabling fast and reliable execution and preventing the need to check in node_modules.
In order to make these commands available in the action output, you'll need to define them in `pyproject.toml` using a section appropriate for the particular tools you are using in the project. You can specify all, some, or none, depending on what you need available.

Actions are run from GitHub repos. Packaging the action will create a packaged action in the dist folder.
This action will pull the command from the first entry it finds in any of the following sections in `pyproject.toml`:
- `[tool.tasks]`
- `[tool.pdm.scripts]`
- `[tool.poe.tasks]`
- `[tool.invoke.tasks]`

Run prepare

```bash
npm run prepare
```
### Where is the support for `[tool.poetry.scripts]`?
Unfortunately, [Poetry scripts](https://python-poetry.org/docs/pyproject/#scripts) is for specifying commands that are made available to consumers of a package. It isn't meant for CI/CD or developer operations and doesn't meet those needs, primarily because any scripts you define in this section will be added as executable shortcuts to your virtual environment or any virtual environment your package is installed into.
If you are using Poetry and want to take advantage of this feature from this action, use the catch-all `[tool.tasks]` section. Alternatively, you
can leverage a task runner tool like [Poe](https://github.com/nat-n/poethepoet) or [Invoke](https://www.pyinvoke.org/)

Since the packaged index.js is run from the dist folder.
*If Poetry ever adds support for internal project (CI/CD/Dev) commands separate from published commands, then it will be added to this action.*

```bash
git add dist
```

## Create a release branch
### Relevant References / Discussions
https://discuss.python.org/t/a-new-pep-to-specify-dev-scripts-and-or-dev-scripts-providers-in-pyproject-toml/11457

Users shouldn't consume the action from master since that would be latest code and actions can break compatibility between major versions.
https://discuss.python.org/t/proposal-for-tests-entry-point-in-pyproject-toml/2077

Checkin to the v1 release branch
https://stackoverflow.com/questions/70386944/how-should-poetry-scripts-used-in-the-build-process-be-stored-in-the-project

```bash
git checkout -b v1
git commit -a -m "v1 release"
```
https://github.com/python-poetry/poetry/issues/3386

```bash
git push origin v1
```

Note: We recommend using the `--license` option for ncc, which will create a license file for all of the production node modules used in your project.

Your action is now published! :rocket:

See the [versioning documentation](https://github.com/actions/toolkit/blob/master/docs/action-versioning.md)

## Usage

You can now consume the action by referencing the v1 branch

```yaml
uses: actions/javascript-action@v1
with:
milliseconds: 1000
```

See the [actions tab](https://github.com/actions/javascript-action/actions) for runs of this action! :rocket:
## Example Workflow
[.github/workflows/examples/example.yml](.github/workflows/examples/example.yml)
65 changes: 65 additions & 0 deletions __snapshots__/find-python-projects.test.js.snap
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`find-python-projects finds projects 1`] = `
{
"packageable-projects": [
{
"buildBackend": "poetry.core.masonry.api",
"directory": "test-fixtures/multi-project/project-2",
"installCommand": "poetry install",
"name": "sub-project-2",
"packageCommand": "poetry bundle venv",
"path": "test-fixtures/multi-project/project-2/pyproject.toml",
"pythonVersion": "^3.9",
"testCommand": "poetry run pytest",
},
],
"paths": [
"test-fixtures/multi-project/project-1/pyproject.toml",
"test-fixtures/multi-project/project-2/pyproject.toml",
"test-fixtures/multi-project/project-4/pyproject.toml",
],
"projects": [
{
"buildBackend": "poetry.core.masonry.api",
"directory": "test-fixtures/multi-project/project-1",
"installCommand": "poetry install",
"name": "sub-project-1",
"packageCommand": null,
"path": "test-fixtures/multi-project/project-1/pyproject.toml",
"pythonVersion": "^3.9",
"testCommand": null,
},
{
"buildBackend": "poetry.core.masonry.api",
"directory": "test-fixtures/multi-project/project-2",
"installCommand": "poetry install",
"name": "sub-project-2",
"packageCommand": "poetry bundle venv",
"path": "test-fixtures/multi-project/project-2/pyproject.toml",
"pythonVersion": "^3.9",
"testCommand": "poetry run pytest",
},
{
"directory": "test-fixtures/multi-project/project-4",
"name": "sub-project-4",
"packageCommand": null,
"path": "test-fixtures/multi-project/project-4/pyproject.toml",
"pythonVersion": "^3.9",
"testCommand": null,
},
],
"testable-projects": [
{
"buildBackend": "poetry.core.masonry.api",
"directory": "test-fixtures/multi-project/project-2",
"installCommand": "poetry install",
"name": "sub-project-2",
"packageCommand": "poetry bundle venv",
"path": "test-fixtures/multi-project/project-2/pyproject.toml",
"pythonVersion": "^3.9",
"testCommand": "poetry run pytest",
},
],
}
`;
22 changes: 11 additions & 11 deletions action.yml
Original file line number Diff line number Diff line change
@@ -1,25 +1,25 @@
name: 'Find Python Projects'
description: 'TODO Description'
name: Find Python Projects
description: Discovers Python sub-projects in a monorepo workspace project

inputs:
root-path:
description: 'TODO Description'
root-dir:
description: Directory root for where to begin recursively searching for projects. Python projects contained in this directory or lower will be discovered.
required: false
default: '.'

outputs:
projects:
description: 'TODO'

paths:
description: 'TODO: JSON array of projects paths'
description: JSON array of found project path strings

projects:
description: JSON array of all found projects (`project` object)

testable-projects:
description: 'TODO'
description: JSON array of all found projects (`project` object) that implement a `test` command

packageable-projects:
description: 'TODO'
description: JSON array of all found projects (`project` object) that implement a `package` command

runs:
using: 'node16'
using: 'node20'
main: 'dist/index.js'
Loading