Skip to content

Commit

Permalink
add dependabot guidelines to readme (#218)
Browse files Browse the repository at this point in the history
Co-authored-by: Anthony Burdi <anthonyburdi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
  • Loading branch information
3 people committed Apr 22, 2024
1 parent 82270d4 commit da00e1d
Showing 1 changed file with 21 additions and 9 deletions.
30 changes: 21 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ invoke deps

#### Updating `poetry.lock` dependencies

The dependencies installed in our CI and the docker build step are determined by the [poetry.lock file](https://python-poetry.org/docs/basic-usage/#installing-with-poetrylock).
The dependencies installed in our CI and the Docker build step are determined by the [poetry.lock file](https://python-poetry.org/docs/basic-usage/#installing-with-poetrylock).

[To update only a specific dependency](https://python-poetry.org/docs/cli/#update) (such as `great_expectations`) ...

Expand Down Expand Up @@ -127,7 +127,7 @@ Now go into GX Cloud and issue commands for the GX Agent to run, such as generat
#### Example Data

The contents from [/examples/agent/data](/examples/agent/data/) will be copied to `/data` for the docker container.
The contents from [/examples/agent/data](/examples/agent/data/) will be copied to `/data` for the Docker container.

#### Adding an action to the agent

Expand All @@ -141,7 +141,7 @@ Note: The agent is core-version specific but this registration mechanism allows

#### Versioning

This is the version that will be used for the docker image tag as well.
This is the version that will be used for the Docker image tag as well.

_Standard Release_:
The versioning scheme is `YYYYMMDD.{release_number}` where:
Expand All @@ -153,7 +153,7 @@ The versioning scheme is `YYYYMMDD.{release_number}` where:
For example: `20240403.0`

_Pre-release_:
The versioing scheme is `YYYYMMDD.{release_number}.dev{dev_number}`
The versioning scheme is `YYYYMMDD.{release_number}.dev{dev_number}`

- the date is the date of the release
- the dev number starts at 0 for the first pre-release of the day
Expand All @@ -176,7 +176,7 @@ There can be days with no standard releases, only pre-releases or days with no p

Pre-releases are completed automatically with each merge to the `main` branch.
The version is updated in `pyproject.toml` and a pre-release is created on PyPi.
A new docker tag will also be generated and pushed to [Docker Hub](https://hub.docker.com/r/greatexpectations/agent)
A new Docker tag will also be generated and pushed to [Docker Hub](https://hub.docker.com/r/greatexpectations/agent)

**Manual Pre-releases**

Expand All @@ -189,7 +189,7 @@ invoke pre-release
```

This will create a new pre-release version. On the next merge to `main`, the release will be uploaded to PyPi.
A new docker tag will also be generated and pushed to [Docker Hub](https://hub.docker.com/r/greatexpectations/agent)
A new Docker tag will also be generated and pushed to [Docker Hub](https://hub.docker.com/r/greatexpectations/agent)

#### Releases

Expand All @@ -203,14 +203,26 @@ invoke release
```

This will create a new release version. On the next merge to `main`, the release will be uploaded to PyPi.
A new docker tag will also be generated and pushed to [Docker Hub](https://hub.docker.com/r/greatexpectations/agent)
A new Docker tag will also be generated and pushed to [Docker Hub](https://hub.docker.com/r/greatexpectations/agent). In addition, releases will be tagged with `stable` and `latest` tags.

#### Github Workflow for releasing
#### GitHub Workflow for releasing

We use the GitHub Actions workflow to automate the release and pre-release process. There are two workflows involved:

1. [CI](./.github/workflows/ci.yml) - This workflow runs on each pull request and will update the version in `pyproject.toml` to the pre-release version if the version is not already manually updated in the PR. It will also run the tests and linting.

2. [Containerize Agent](./.github/workflows/containerize-agent.yml) - This workflows runs on merge with `main` and will create a new docker image and push it to Docker Hub and PyPi. It uses the version in `pyproject.toml`.
2. [Containerize Agent](./.github/workflows/containerize-agent.yml) - This workflows runs on merge with `main` and will create a new Docker image and push it to Docker Hub and PyPi. It uses the version in `pyproject.toml`.

A visual representation of the workflow is shown [here](./.github/workflows/agent_release_workflows.png)

### Dependabot and Releases/Pre-releases
GitHub's Dependabot regularly checks our dependencies for vulnerabilty-based updates and proposes PRs to update dependency version numbers accordingly.

Dependabot may only update the `poetry.lock` file. Before merging Dependabot suggestions, we should also ensure that `pyproject.toml` is aligned with version locked in the `poetry.lock` file by following the instructions above at [Updating `poetry.lock` dependencies](#updating-poetry.lock-dependencies).

Note: if Dependabot suggests an update to a tool in the `[tool.poetry.group.dev.dependencies]` group in `pyproject.toml`, these changes can be merged in a pre-release version (i.e., a standard release is not required). While doing this, make sure any version references in the pre-commit config `.pre-commit-config.yaml` are kept in sync (e.g., ruff).

For other dependency updates, a new release should be orchestrated. This includes updates in the following sections:

- `[tool.poetry.dependencies]`
- `[tool.poetry.group.*.dependencies]` where `*` is the name of the group (not including the `dev` group)

0 comments on commit da00e1d

Please sign in to comment.