Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source Trello: Enable in Cloud #24045

Merged
merged 10 commits into from
Mar 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2041,7 +2041,7 @@
- name: Trello
sourceDefinitionId: 8da67652-004c-11ec-9a03-0242ac130003
dockerRepository: airbyte/source-trello
dockerImageTag: 0.1.6
dockerImageTag: 0.2.0
documentationUrl: https://docs.airbyte.com/integrations/sources/trello
icon: trello.svg
sourceType: api
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15609,7 +15609,7 @@
supportsNormalization: false
supportsDBT: false
supported_destination_sync_modes: []
- dockerImage: "airbyte/source-trello:0.1.6"
- dockerImage: "airbyte/source-trello:0.2.0"
spec:
documentationUrl: "https://docs.airbyte.com/integrations/sources/trello"
connectionSpecification:
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-trello/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -29,5 +29,5 @@ COPY source_trello ./source_trello
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.6
grubberr marked this conversation as resolved.
Show resolved Hide resolved
LABEL io.airbyte.version=0.2.0
LABEL io.airbyte.name=airbyte/source-trello
Original file line number Diff line number Diff line change
@@ -1,26 +1,34 @@
# See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference)
# for more information about how to configure these tests
connector_image: airbyte/source-trello:dev
tests:
acceptance_tests:
spec:
- spec_path: "source_trello/spec.json"
tests:
- spec_path: "source_trello/spec.json"
connection:
- config_path: "secrets/config.json"
status: "succeed"
- config_path: "integration_tests/invalid_config.json"
status: "failed"
tests:
- config_path: "secrets/config.json"
status: "succeed"
- config_path: "integration_tests/invalid_config.json"
status: "failed"
discovery:
- config_path: "secrets/config.json"
tests:
- config_path: "secrets/config.json"
backward_compatibility_tests_config:
disable_for_version: 0.1.6
basic_read:
- config_path: "secrets/config.json"
configured_catalog_path: "integration_tests/configured_catalog.json"
empty_streams: []
tests:
- config_path: "secrets/config.json"
empty_streams: []
incremental:
- config_path: "secrets/config.json"
configured_catalog_path: "integration_tests/configured_catalog.json"
future_state_path: "integration_tests/abnormal_state.json"
cursor_paths:
actions: ["date"]
tests:
- config_path: "secrets/config.json"
configured_catalog_path: "integration_tests/configured_catalog.json"
future_state:
future_state_path: "integration_tests/abnormal_state.json"
cursor_paths:
actions: ["date"]
full_refresh:
- config_path: "secrets/config.json"
configured_catalog_path: "integration_tests/configured_catalog.json"
tests:
- config_path: "secrets/config.json"
configured_catalog_path: "integration_tests/configured_catalog.json"
4 changes: 1 addition & 3 deletions airbyte-integrations/connectors/source-trello/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,7 @@

from setuptools import find_packages, setup

MAIN_REQUIREMENTS = [
"airbyte-cdk~=0.1",
]
MAIN_REQUIREMENTS = ["airbyte-cdk~=0.1", "requests-oauthlib"]

TEST_REQUIREMENTS = [
"pytest~=6.1",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@
"type": ["null", "string"]
},
"ixUpdate": {
"type": ["null", "integer"]
"type": ["null", "string"]
},
"enterpriseOwned": {
"type": ["null", "boolean"]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -208,19 +208,10 @@
}
},
"idChecklists": {
"anyOf": [
{
"type": ["null", "array"],
"items": {
"additionalProperties": false,
"properties": {
"id": {
"type": ["null", "string"]
}
}
}
}
]
"type": ["null", "array"],
"items": {
"type": ["null", "string"]
}
},
"idMembers": {
"type": ["null", "array"],
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from airbyte_cdk.sources import AbstractSource
from airbyte_cdk.sources.streams import Stream
from airbyte_cdk.sources.streams.http import HttpStream
from airbyte_cdk.sources.streams.http.auth import HttpAuthenticator
from requests_oauthlib import OAuth1

from .utils import TrelloRequestRateLimits as balancer

Expand Down Expand Up @@ -167,38 +167,19 @@ def path(self, stream_slice: Mapping[str, Any] = None, **kwargs) -> str:
return f"boards/{stream_slice['id']}/actions"


class TrelloAuthenticator(HttpAuthenticator):
"""
Generate auth header for start making requests from API token and API key.
"""

def __init__(
self,
token: str,
key: str,
auth_header: str = "Authorization",
key_header: str = "oauth_consumer_key",
token_header: str = "oauth_token",
):
self.auth_header = auth_header
self.key_header = key_header
self.token_header = token_header
self._key = key
self._token = token

def get_auth_header(self) -> Mapping[str, Any]:
return {self.auth_header: f'OAuth {self.key_header}="{self._key}", {self.token_header}="{self._token}"'}


class SourceTrello(AbstractSource):
"""
Source Trello fetch date from web-based, Kanban-style, list-making application.
"""

@staticmethod
def _get_authenticator(config: dict) -> TrelloAuthenticator:
key, token = config["key"], config["token"]
return TrelloAuthenticator(token=token, key=key)
def _get_authenticator(config: dict):
return OAuth1(
client_key=config["client_id"],
client_secret=config["client_secret"],
resource_owner_key=config["token"],
resource_owner_secret=config["key"],
)

def check_connection(self, logger, config) -> Tuple[bool, any]:
"""
Expand All @@ -210,7 +191,7 @@ def check_connection(self, logger, config) -> Tuple[bool, any]:

authenticator = self._get_authenticator(config)

response = requests.get(url, headers=authenticator.get_auth_header())
response = requests.get(url, auth=authenticator)
response.raise_for_status()
available_boards = {row.get("id") for row in response.json()}
for board_id in config.get("board_ids", []):
Expand Down
2 changes: 1 addition & 1 deletion connectors.md
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,7 @@
| **TikTok Marketing** | <img alt="TikTok Marketing icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config/init/src/main/resources/icons/tiktok.svg" height="30" height="30"/> | Source | airbyte/source-tiktok-marketing:2.0.4 | generally_available | [link](https://docs.airbyte.com/integrations/sources/tiktok-marketing) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-tiktok-marketing) | <small>`4bfac00d-ce15-44ff-95b9-9e3c3e8fbd35`</small> |
| **Timely** | <img alt="Timely icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config/init/src/main/resources/icons/timely.svg" height="30" height="30"/> | Source | airbyte/source-timely:0.1.0 | alpha | [link](https://docs.airbyte.com/integrations/sources/timely) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-timely) | <small>`bc617b5f-1b9e-4a2d-bebe-782fd454a771`</small> |
| **Toggl** | <img alt="Toggl icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config/init/src/main/resources/icons/toggl.svg" height="30" height="30"/> | Source | airbyte/source-toggl:0.1.0 | alpha | [link](https://docs.airbyte.com/integrations/sources/toggl) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-toggl) | <small>`7e7c844f-2300-4342-b7d3-6dd7992593cd`</small> |
| **Trello** | <img alt="Trello icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config/init/src/main/resources/icons/trello.svg" height="30" height="30"/> | Source | airbyte/source-trello:0.1.6 | alpha | [link](https://docs.airbyte.com/integrations/sources/trello) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-trello) | <small>`8da67652-004c-11ec-9a03-0242ac130003`</small> |
| **Trello** | <img alt="Trello icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config/init/src/main/resources/icons/trello.svg" height="30" height="30"/> | Source | airbyte/source-trello:0.2.0 | alpha | [link](https://docs.airbyte.com/integrations/sources/trello) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-trello) | <small>`8da67652-004c-11ec-9a03-0242ac130003`</small> |
| **Twilio** | <img alt="Twilio icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config/init/src/main/resources/icons/twilio.svg" height="30" height="30"/> | Source | airbyte/source-twilio:0.1.16 | generally_available | [link](https://docs.airbyte.com/integrations/sources/twilio) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-twilio) | <small>`b9dc6155-672e-42ea-b10d-9f1f1fb95ab1`</small> |
| **Twilio Taskrouter** | <img alt="Twilio Taskrouter icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config/init/src/main/resources/icons/twilio.svg" height="30" height="30"/> | Source | airbyte/source-twilio-taskrouter:0.1.0 | alpha | [link](https://docs.airbyte.com/integrations/sources/twilio-taskrouter) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-twilio-taskrouter) | <small>`2446953b-b794-429b-a9b3-c821ba992a48`</small> |
| **Twitter** | <img alt="Twitter icon" src="https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-config/init/src/main/resources/icons/twitter.svg" height="30" height="30"/> | Source | airbyte/source-twitter:0.1.2 | beta | [link](https://docs.airbyte.com/integrations/sources/twitter) | [code](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-twitter) | <small>`d7fd4f40-5e5a-4b8b-918f-a73077f8c131`</small> |
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/trello.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ Please read [How to get your APIs Token and Key](https://developer.atlassian.com

| Version | Date | Pull Request | Subject |
| :--- | :--- | :--- | :--- |
| 0.2.0 | 2023-03-15 | [24045](https://github.com/airbytehq/airbyte/pull/24045) | Fix schema for boards and cards streams |
| 0.1.6 | 2021-12-28 | [8628](https://github.com/airbytehq/airbyte/pull/8628) | Updated fields in source-connector specifications |
| 0.1.3 | 2021-11-25 | [8183](https://github.com/airbytehq/airbyte/pull/8183) | Enable specifying board ids in configuration |
| 0.1.2 | 2021-11-08 | [7499](https://github.com/airbytehq/airbyte/pull/7499) | Remove base-python dependencies |
Expand Down