Skip to content

Commit

Permalink
airbyte-ci: remove connector format command
Browse files Browse the repository at this point in the history
  • Loading branch information
alafanechere committed Oct 14, 2023
1 parent 896fa4b commit a228c1f
Show file tree
Hide file tree
Showing 6 changed files with 8 additions and 258 deletions.
36 changes: 5 additions & 31 deletions airbyte-ci/connectors/pipelines/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,6 @@ At this point you can run `airbyte-ci` commands from the root of the repository.
- [`connectors` command subgroup](#connectors-command-subgroup)
* [Options](#options-1)
- [`connectors list` command](#connectors-list-command)
- [`connectors format` command](#connectors-format-command)
- [`connectors test` command](#connectors-test-command)
* [Examples](#examples-)
* [What it runs](#what-it-runs-)
Expand Down Expand Up @@ -169,32 +168,6 @@ List connectors with multiple filters:

`airbyte-ci connectors --language=low-code --support-level=certified list`

### <a id="connectors-list-command"></a>`connectors format` command
Run a code formatter on one or multiple connectors.

For Python connectors we run the following tools, using the configuration defined in `pyproject.toml`:
* `black` for code formatting
* `isort` for import sorting
* `licenseheaders` for adding license headers

For Java connectors we run `./gradlew format`.

In local CLI execution the formatted code is exported back the local repository. No commit or push is performed.
In CI execution the formatted code is pushed to the remote branch. One format commit per connector.

#### Examples
Format a specific connector:

`airbyte-ci connectors --name=source-pokeapi format`

Format all Python connectors:

`airbyte-ci connectors --language=python format`

Format all connectors modified on the current branch:

`airbyte-ci connectors --modified format`


### <a id="connectors-test-command"></a>`connectors test` command
Run a test pipeline for one or multiple connectors.
Expand Down Expand Up @@ -395,10 +368,11 @@ This command runs the Python tests for a airbyte-ci poetry package.
## Changelog
| Version | PR | Description |
| ------- | ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
| 1.7.2 | [#31343](https://github.com/airbytehq/airbyte/pull/31343) | Bind Pytest integration tests to a dockerhost. |
| 1.7.1 | [#31332](https://github.com/airbytehq/airbyte/pull/31332) | Disable Gradle step caching on source-postgres. |
| 1.7.0 | [#30526](https://github.com/airbytehq/airbyte/pull/30526) | Implement pre/post install hooks support. |
| 1.6.0 | [#30474](https://github.com/airbytehq/airbyte/pull/30474) | Test connector inside their containers. |
| 2.0.0 | [#TBD](https://github.com/airbytehq/airbyte/pull/TBD) | Remove `airbyte-ci connectors format` command. |
| 1.7.2 | [#31343](https://github.com/airbytehq/airbyte/pull/31343) | Bind Pytest integration tests to a dockerhost. |
| 1.7.1 | [#31332](https://github.com/airbytehq/airbyte/pull/31332) | Disable Gradle step caching on source-postgres. |
| 1.7.0 | [#30526](https://github.com/airbytehq/airbyte/pull/30526) | Implement pre/post install hooks support. |
| 1.6.0 | [#30474](https://github.com/airbytehq/airbyte/pull/30474) | Test connector inside their containers. |
| 1.5.1 | [#31227](https://github.com/airbytehq/airbyte/pull/31227) | Use python 3.11 in amazoncorretto-bazed gradle containers, run 'test' gradle task instead of 'check'. |
| 1.5.0 | [#30456](https://github.com/airbytehq/airbyte/pull/30456) | Start building Python connectors using our base images. |
| 1.4.6 | [ #31087](https://github.com/airbytehq/airbyte/pull/31087) | Throw error if airbyte-ci tools is out of date |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,13 @@
from pipelines.bases import ConnectorWithModifiedFiles
from pipelines.builds import run_connector_build_pipeline
from pipelines.contexts import ConnectorContext, ContextState, PublishConnectorContext
from pipelines.format import run_connectors_format_pipelines
from pipelines.github import update_global_commit_status_check_for_tests
from pipelines.pipelines.connectors import run_connectors_pipelines
from pipelines.publish import reorder_contexts, run_connector_publish_pipeline
from pipelines.tests import run_connector_test_pipeline
from pipelines.utils import DaggerPipelineCommand, get_connector_modified_files, get_modified_connectors
from rich.table import Table
from rich.text import Text

# HELPERS

Expand Down Expand Up @@ -508,45 +509,6 @@ def list(
return True


@connectors.command(name="format", cls=DaggerPipelineCommand, help="Autoformat connector code.")
@click.pass_context
def format_code(ctx: click.Context) -> bool:
connectors_contexts = [
ConnectorContext(
pipeline_name=f"Format connector {connector.technical_name}",
connector=connector,
is_local=ctx.obj["is_local"],
git_branch=ctx.obj["git_branch"],
git_revision=ctx.obj["git_revision"],
ci_report_bucket=ctx.obj["ci_report_bucket_name"],
report_output_prefix=ctx.obj["report_output_prefix"],
use_remote_secrets=ctx.obj["use_remote_secrets"],
gha_workflow_run_url=ctx.obj.get("gha_workflow_run_url"),
dagger_logs_url=ctx.obj.get("dagger_logs_url"),
pipeline_start_timestamp=ctx.obj.get("pipeline_start_timestamp"),
ci_context=ctx.obj.get("ci_context"),
ci_gcs_credentials=ctx.obj["ci_gcs_credentials"],
ci_git_user=ctx.obj["ci_git_user"],
ci_github_access_token=ctx.obj["ci_github_access_token"],
pull_request=ctx.obj.get("pull_request"),
should_save_report=False,
)
for connector in ctx.obj["selected_connectors_with_modified_files"]
]

anyio.run(
run_connectors_format_pipelines,
connectors_contexts,
ctx.obj["ci_git_user"],
ctx.obj["ci_github_access_token"],
ctx.obj["git_branch"],
ctx.obj["is_local"],
ctx.obj["execute_timeout"],
)

return True


def log_selected_connectors(selected_connectors_with_modified_files: List[ConnectorWithModifiedFiles]) -> None:
if selected_connectors_with_modified_files:
selected_connectors_names = [c.technical_name for c in selected_connectors_with_modified_files]
Expand Down
101 changes: 0 additions & 101 deletions airbyte-ci/connectors/pipelines/pipelines/format/__init__.py

This file was deleted.

This file was deleted.

This file was deleted.

2 changes: 1 addition & 1 deletion airbyte-ci/connectors/pipelines/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "pipelines"
version = "1.7.2"
version = "2.0.0"
description = "Packaged maintained by the connector operations team to perform CI for connectors' pipelines"
authors = ["Airbyte <contact@airbyte.io>"]

Expand Down

0 comments on commit a228c1f

Please sign in to comment.