Skip to content

Commit

Permalink
airbyte-ci: remove connector format command
Browse files Browse the repository at this point in the history
  • Loading branch information
alafanechere committed Oct 17, 2023
1 parent cf733f1 commit a620d58
Show file tree
Hide file tree
Showing 7 changed files with 6 additions and 259 deletions.
36 changes: 5 additions & 31 deletions airbyte-ci/connectors/pipelines/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,6 @@ At this point you can run `airbyte-ci` commands from the root of the repository.
- [`connectors` command subgroup](#connectors-command-subgroup)
* [Options](#options-1)
- [`connectors list` command](#connectors-list-command)
- [`connectors format` command](#connectors-format-command)
- [`connectors test` command](#connectors-test-command)
* [Examples](#examples-)
* [What it runs](#what-it-runs-)
Expand Down Expand Up @@ -171,32 +170,6 @@ List connectors with multiple filters:

`airbyte-ci connectors --language=low-code --support-level=certified list`

### <a id="connectors-list-command"></a>`connectors format` command
Run a code formatter on one or multiple connectors.

For Python connectors we run the following tools, using the configuration defined in `pyproject.toml`:
* `black` for code formatting
* `isort` for import sorting
* `licenseheaders` for adding license headers

For Java connectors we run `./gradlew format`.

In local CLI execution the formatted code is exported back the local repository. No commit or push is performed.
In CI execution the formatted code is pushed to the remote branch. One format commit per connector.

#### Examples
Format a specific connector:

`airbyte-ci connectors --name=source-pokeapi format`

Format all Python connectors:

`airbyte-ci connectors --language=python format`

Format all connectors modified on the current branch:

`airbyte-ci connectors --modified format`


### <a id="connectors-test-command"></a>`connectors test` command
Run a test pipeline for one or multiple connectors.
Expand Down Expand Up @@ -382,9 +355,9 @@ Make a connector using a Dockerfile migrate to the base image by:
Migrate source-openweather to use the base image: `airbyte-ci connectors --name=source-openweather migrate-to-base-image`

### Arguments
| Argument | Description |
| --------------------- | ---------------------------------------------------------------------- |
| `PULL_REQUEST_NUMBER` | The GitHub pull request number, used in the changelog entry |
| Argument | Description |
| --------------------- | ----------------------------------------------------------- |
| `PULL_REQUEST_NUMBER` | The GitHub pull request number, used in the changelog entry |

### <a id="metadata-validate-command-subgroup"></a>`metadata` command subgroup

Expand Down Expand Up @@ -424,7 +397,8 @@ This command runs the Python tests for a airbyte-ci poetry package.

## Changelog
| Version | PR | Description |
|---------| ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
| ------- | ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
| 2.0.0 | [#31424](https://github.com/airbytehq/airbyte/pull/31424) | Remove `airbyte-ci connectors format` command. |
| 1.9.4 | [#31478](https://github.com/airbytehq/airbyte/pull/31478) | Fix running tests for connector-ops package. |
| 1.9.3 | [#31457](https://github.com/airbytehq/airbyte/pull/31457) | Improve the connector documentation for connectors migrated to our base image. |
| 1.9.2 | [#31426](https://github.com/airbytehq/airbyte/pull/31426) | Concurrent execution of java connectors tests. |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@
)
from pipelines.connector_changes.version_bump import run_connector_version_bump_pipeline
from pipelines.contexts import ConnectorContext, ContextState, PublishConnectorContext
from pipelines.format import run_connector_format_pipeline
from pipelines.github import update_global_commit_status_check_for_tests
from pipelines.pipelines.connectors import run_connectors_pipelines
from pipelines.publish import reorder_contexts, run_connector_publish_pipeline
Expand Down Expand Up @@ -515,45 +514,6 @@ def list(
return True


@connectors.command(name="format", cls=DaggerPipelineCommand, help="Autoformat connector code.")
@click.pass_context
def format_code(ctx: click.Context) -> bool:
connectors_contexts = [
ConnectorContext(
pipeline_name=f"Format connector {connector.technical_name}",
connector=connector,
is_local=ctx.obj["is_local"],
git_branch=ctx.obj["git_branch"],
git_revision=ctx.obj["git_revision"],
ci_report_bucket=ctx.obj["ci_report_bucket_name"],
report_output_prefix=ctx.obj["report_output_prefix"],
use_remote_secrets=ctx.obj["use_remote_secrets"],
gha_workflow_run_url=ctx.obj.get("gha_workflow_run_url"),
dagger_logs_url=ctx.obj.get("dagger_logs_url"),
pipeline_start_timestamp=ctx.obj.get("pipeline_start_timestamp"),
ci_context=ctx.obj.get("ci_context"),
ci_gcs_credentials=ctx.obj["ci_gcs_credentials"],
ci_git_user=ctx.obj["ci_git_user"],
ci_github_access_token=ctx.obj["ci_github_access_token"],
pull_request=ctx.obj.get("pull_request"),
should_save_report=False,
)
for connector in ctx.obj["selected_connectors_with_modified_files"]
]

anyio.run(
run_connectors_pipelines,
connectors_contexts,
run_connector_format_pipeline,
"Format connectors pipeline",
ctx.obj["concurrency"],
ctx.obj["dagger_logs_path"],
ctx.obj["execute_timeout"],
)

return True


@connectors.command(cls=DaggerPipelineCommand, help="Upgrades the base image version used by the selected connectors..")
@click.option("--set-if-not-exists", default=True)
@click.option(
Expand Down
101 changes: 0 additions & 101 deletions airbyte-ci/connectors/pipelines/pipelines/format/__init__.py

This file was deleted.

This file was deleted.

This file was deleted.

2 changes: 1 addition & 1 deletion airbyte-ci/connectors/pipelines/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "pipelines"
version = "1.9.4"
version = "2.0.0"
description = "Packaged maintained by the connector operations team to perform CI for connectors' pipelines"
authors = ["Airbyte <contact@airbyte.io>"]

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -266,7 +266,6 @@ def click_context_obj():
"test",
],
),
(connectors.format_code, []),
(connectors.build, []),
],
)
Expand Down

0 comments on commit a620d58

Please sign in to comment.