diff --git a/airbyte-integrations/connectors/source-monday/Dockerfile b/airbyte-integrations/connectors/source-monday/Dockerfile deleted file mode 100644 index e3e950b0cfefb..0000000000000 --- a/airbyte-integrations/connectors/source-monday/Dockerfile +++ /dev/null @@ -1,38 +0,0 @@ -FROM python:3.9.11-alpine3.15 as base - -# build and load all requirements -FROM base as builder -WORKDIR /airbyte/integration_code - -# upgrade pip to the latest version -RUN apk --no-cache upgrade \ - && pip install --upgrade pip \ - && apk --no-cache add tzdata build-base - - -COPY setup.py ./ -# install necessary packages to a temporary folder -RUN pip install --prefix=/install . - -# build a clean environment -FROM base -WORKDIR /airbyte/integration_code - -# copy all loaded and built libraries to a pure basic image -COPY --from=builder /install /usr/local -# add default timezone settings -COPY --from=builder /usr/share/zoneinfo/Etc/UTC /etc/localtime -RUN echo "Etc/UTC" > /etc/timezone - -# bash is installed for more convenient debugging. -RUN apk --no-cache add bash - -# copy payload code only -COPY main.py ./ -COPY source_monday ./source_monday - -ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py" -ENTRYPOINT ["python", "/airbyte/integration_code/main.py"] - -LABEL io.airbyte.version=1.1.3 -LABEL io.airbyte.name=airbyte/source-monday diff --git a/airbyte-integrations/connectors/source-monday/README.md b/airbyte-integrations/connectors/source-monday/README.md index 88b2490a0940d..b0e9c8b4ffc40 100644 --- a/airbyte-integrations/connectors/source-monday/README.md +++ b/airbyte-integrations/connectors/source-monday/README.md @@ -6,23 +6,28 @@ For information about how to use this connector within Airbyte, see [the documen ## Local development ### Prerequisites + **To iterate on this connector, make sure to complete this prerequisites section.** #### Minimum Python version required `= 3.7.0` #### Build & Activate Virtual Environment and install dependencies + From this connector directory, create a virtual environment: -``` + +```bash python -m venv .venv ``` This will generate a virtualenv for this module in `.venv/`. Make sure this venv is active in your development environment of choice. To activate it from the terminal, run: -``` + +```bash source .venv/bin/activate pip install -r requirements.txt pip install '.[tests]' ``` + If you are in an IDE, follow your IDE's instructions to activate the virtualenv. Note that while we are installing dependencies from `requirements.txt`, you should only edit `setup.py` for your dependencies. `requirements.txt` is @@ -31,6 +36,7 @@ If this is mumbo jumbo to you, don't worry about it, just put your deps in `setu should work as you expect. #### Create credentials + **If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.io/integrations/sources/monday) to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `source_monday/spec.json` file. Note that any directory named `secrets` is gitignored across the entire Airbyte repo, so there is no danger of accidentally checking in sensitive information. @@ -40,7 +46,8 @@ See `integration_tests/sample_config.json` for a sample config file. and place them into `secrets/config.json`. ### Locally running the connector -``` + +```bash python main.py spec python main.py check --config secrets/config.json python main.py discover --config secrets/config.json @@ -49,23 +56,82 @@ python main.py read --config secrets/config.json --catalog integration_tests/con ### Locally running the connector docker image +#### Use `airbyte-ci` to build your connector + +The Airbyte way of building this connector is to use our `airbyte-ci` tool. +You can follow install instructions [here](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md#L1). +Then running the following command will build your connector: -#### Build -**Via [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md) (recommended):** ```bash airbyte-ci connectors --name=source-monday build ``` -An image will be built with the tag `airbyte/source-monday:dev`. +Once the command is done, you will find your connector image in your local docker registry: `airbyte/source-monday:dev`. + +##### Customizing our build process + +When contributing on our connector you might need to customize the build process to add a system dependency or set an env var. +You can customize our build process by adding a `build_customization.py` module to your connector. +This module should contain a `pre_connector_install` and `post_connector_install` async function that will mutate the base image and the connector container respectively. +It will be imported at runtime by our build process and the functions will be called if they exist. + +Here is an example of a `build_customization.py` module: + +```python +from __future__ import annotations + +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + # Feel free to check the dagger documentation for more information on the Container object and its methods. + # https://dagger-io.readthedocs.io/en/sdk-python-v0.6.4/ + from dagger import Container + + +async def pre_connector_install(base_image_container: Container) -> Container: + return await base_image_container.with_env_variable("MY_PRE_BUILD_ENV_VAR", "my_pre_build_env_var_value") + +async def post_connector_install(connector_container: Container) -> Container: + return await connector_container.with_env_variable("MY_POST_BUILD_ENV_VAR", "my_post_build_env_var_value") +``` + +#### Build your own connector image + +This connector is built using our dynamic built process in `airbyte-ci`. +The base image used to build it is defined within the metadata.yaml file under the `connectorBuildOptions`. +The build logic is defined using [Dagger](https://dagger.io/) [here](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/pipelines/builds/python_connectors.py). +It does not rely on a Dockerfile. + +If you would like to patch our connector and build your own a simple approach would be to: + +1. Create your own Dockerfile based on the latest version of the connector image. + +```Dockerfile +FROM airbyte/source-monday:latest + +COPY . ./airbyte/integration_code +RUN pip install ./airbyte/integration_code + +# The entrypoint and default env vars are already set in the base image +# ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py" +# ENTRYPOINT ["python", "/airbyte/integration_code/main.py"] +``` + +Please use this as an example. This is not optimized. + +2. Build your image: -**Via `docker build`:** ```bash docker build -t airbyte/source-monday:dev . +# Running the spec command against your patched connector +docker run airbyte/source-monday:dev spec ``` #### Run + Then run any of the connector commands as follows: -``` + +```bash docker run --rm airbyte/source-monday:dev spec docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-monday:dev check --config /secrets/config.json docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-monday:dev discover --config /secrets/config.json @@ -73,23 +139,30 @@ docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integrat ``` ## Testing + You can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md): + ```bash airbyte-ci connectors --name=source-monday test ``` ### Customizing acceptance Tests + Customize `acceptance-test-config.yml` file to configure tests. See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information. If your connector requires to create or destroy resources for use during acceptance tests create fixtures for it and place them inside integration_tests/acceptance.py. ## Dependency Management + All of your dependencies should go in `setup.py`, NOT `requirements.txt`. The requirements file is only used to connect internal Airbyte dependencies in the monorepo for local development. We split dependencies between two groups, dependencies that are: + * required for your connector to work need to go to `MAIN_REQUIREMENTS` list. * required for the testing need to go to `TEST_REQUIREMENTS` list ### Publishing a new version of the connector + You've checked out the repo, implemented a million dollar feature, and you're ready to share your changes with the world. Now what? + 1. Make sure your changes are passing our test suite: `airbyte-ci connectors --name=source-monday test` 2. Bump the connector version in `metadata.yaml`: increment the `dockerImageTag` value. Please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors). 3. Make sure the `metadata.yaml` content is up to date. @@ -97,4 +170,3 @@ You've checked out the repo, implemented a million dollar feature, and you're re 5. Create a Pull Request: use [our PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#pull-request-title-convention). 6. Pat yourself on the back for being an awesome contributor. 7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master. - diff --git a/airbyte-integrations/connectors/source-monday/integration_tests/expected_records.jsonl b/airbyte-integrations/connectors/source-monday/integration_tests/expected_records.jsonl index d7077351a0ca6..05a950303148d 100644 --- a/airbyte-integrations/connectors/source-monday/integration_tests/expected_records.jsonl +++ b/airbyte-integrations/connectors/source-monday/integration_tests/expected_records.jsonl @@ -1,16 +1,16 @@ {"stream": "items", "data": {"assets": [], "board": {"id": "4635211873"}, "column_values": [{"additional_info": null, "description": null, "id": "person", "text": "", "title": "Person", "type": "multiple-person", "value": null}, {"additional_info": "{\"label\":\"Working on it\",\"color\":\"#fdab3d\",\"changed_at\":\"2019-03-01T17:24:57.321Z\"}", "description": null, "id": "status", "text": "Working on it", "title": "Status", "type": "color", "value": "{\"index\":0,\"post_id\":null,\"changed_at\":\"2019-03-01T17:24:57.321Z\"}"}, {"additional_info": null, "description": null, "id": "date4", "text": "2023-06-11", "title": "Date", "type": "date", "value": "{\"date\":\"2023-06-11\",\"icon\":null,\"changed_at\":\"2023-06-13T13:58:25.871Z\"}"}, {"additional_info": null, "description": null, "id": "tags", "text": "open", "title": "Tags", "type": "tag", "value": "{\"tag_ids\":[19038090]}"}], "created_at": "2023-06-13T13:58:24Z", "creator_id": "36694549", "group": {"id": "topics"}, "id": "4635211945", "name": "Item 1", "parent_item": null, "state": "active", "subscribers": [{"id": 36694549}], "updated_at": "2023-06-15T16:19:37Z", "updates": [{"id": "2223820299"}, {"id": "2223818363"}], "updated_at_int": 1686845977}, "emitted_at": 1690884054247} {"stream": "items", "data": {"assets": [], "board": {"id": "4635211873"}, "column_values": [{"additional_info": null, "description": null, "id": "person", "text": "", "title": "Person", "type": "multiple-person", "value": null}, {"additional_info": "{\"label\":\"Done\",\"color\":\"#00c875\",\"changed_at\":\"2019-03-01T17:28:23.178Z\"}", "description": null, "id": "status", "text": "Done", "title": "Status", "type": "color", "value": "{\"index\":1,\"post_id\":null,\"changed_at\":\"2019-03-01T17:28:23.178Z\"}"}, {"additional_info": null, "description": null, "id": "date4", "text": "2023-06-11", "title": "Date", "type": "date", "value": "{\"date\":\"2023-06-11\",\"icon\":null,\"changed_at\":\"2023-06-13T13:58:25.871Z\"}"}, {"additional_info": null, "description": null, "id": "tags", "text": "closed", "title": "Tags", "type": "tag", "value": "{\"tag_ids\":[19038091]}"}], "created_at": "2023-06-13T13:58:24Z", "creator_id": "36694549", "group": {"id": "topics"}, "id": "4635211964", "name": "Item 2", "parent_item": null, "state": "active", "subscribers": [{"id": 36694549}], "updated_at": "2023-06-13T13:59:36Z", "updates": [], "updated_at_int": 1686664776}, "emitted_at": 1690884054254} {"stream": "items", "data": {"assets": [], "board": {"id": "4635211873"}, "column_values": [{"additional_info": null, "description": null, "id": "person", "text": "", "title": "Person", "type": "multiple-person", "value": null}, {"additional_info": "{\"label\":null,\"color\":\"#c4c4c4\",\"changed_at\":\"2019-03-01T17:25:02.248Z\"}", "description": null, "id": "status", "text": null, "title": "Status", "type": "color", "value": "{\"index\":5,\"post_id\":null,\"changed_at\":\"2019-03-01T17:25:02.248Z\"}"}, {"additional_info": null, "description": null, "id": "date4", "text": "2023-06-13", "title": "Date", "type": "date", "value": "{\"date\":\"2023-06-13\",\"icon\":null,\"changed_at\":\"2023-06-13T13:58:26.291Z\"}"}, {"additional_info": null, "description": null, "id": "tags", "text": "", "title": "Tags", "type": "tag", "value": null}], "created_at": "2023-06-13T13:58:24Z", "creator_id": "36694549", "group": {"id": "topics"}, "id": "4635211977", "name": "Item 3", "parent_item": null, "state": "active", "subscribers": [{"id": 36694549}], "updated_at": "2023-06-13T13:58:26Z", "updates": [], "updated_at_int": 1686664706}, "emitted_at": 1690884054258} -{"stream": "boards", "data": {"board_kind": "public", "type": "board", "columns": [{"archived": false, "description": null, "id": "name", "settings_str": "{}", "title": "Name", "type": "name", "width": 400}, {"archived": false, "description": null, "id": "person", "settings_str": "{}", "title": "Person", "type": "multiple-person", "width": null}, {"archived": false, "description": null, "id": "status", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"Working on it\",\"1\":\"Done\",\"2\":\"Stuck\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "Status", "type": "color", "width": null}, {"archived": false, "description": null, "id": "date4", "settings_str": "{}", "title": "Date", "type": "date", "width": null}, {"archived": false, "description": null, "id": "tags", "settings_str": "{\"hide_footer\":false}", "title": "Tags", "type": "tag", "width": null}], "communication": null, "description": null, "groups": [{"archived": false, "color": "#579bfc", "deleted": false, "id": "topics", "position": "65536", "title": "Group Title"}, {"archived": false, "color": "#a25ddc", "deleted": false, "id": "group_title", "position": "98304", "title": "Group Title"}, {"archived": false, "color": "#808080", "deleted": false, "id": "new_group", "position": "163840.0", "title": "New Group unit board"}], "id": "4635211873", "name": "New Board", "owners": [{"id": 36694549}], "creator": {"id": 36694549}, "permissions": "everyone", "pos": null, "state": "active", "subscribers": [{"id": 36694549}], "tags": [], "top_group": {"id": "topics"}, "updated_at": "2023-06-20T12:12:46Z", "updates": [{"id": "2223820299"}, {"id": "2223818363"}], "views": [], "workspace": {"id": 2845647, "name": "Test workspace", "kind": "open", "description": null}, "updated_at_int": 1687263166}, "emitted_at": 1696447529789} -{"stream": "boards", "data": {"board_kind": "public", "type": "document", "columns": [{"archived": false, "description": null, "id": "name", "settings_str": "{}", "title": "Name", "type": "name", "width": 400}, {"archived": false, "description": null, "id": "files", "settings_str": "{\"hide_footer\":false}", "title": "Files", "type": "file", "width": null}], "communication": null, "description": null, "groups": [{"archived": false, "color": "#579bfc", "deleted": false, "id": "topics", "position": "65536", "title": "Group Title"}], "id": "4634950289", "name": "test doc", "owners": [{"id": 36694549}], "creator": {"id": 36694549}, "permissions": "everyone", "pos": null, "state": "active", "subscribers": [{"id": 36694549}], "tags": [], "top_group": {"id": "topics"}, "updated_at": "2023-06-13T13:28:31Z", "updates": [], "views": [{"id": "103920755", "name": "Table", "settings_str": "{}", "type": "FeatureBoardView", "view_specific_data_str": "{}"}], "workspace": {"id": 2845647, "name": "Test workspace", "kind": "open", "description": null}, "updated_at_int": 1686662911}, "emitted_at": 1696447529793} -{"stream": "boards", "data": {"board_kind": "public", "type": "board", "columns": [{"archived": false, "description": null, "id": "name", "settings_str": "{}", "title": "Name", "type": "name", "width": 380}, {"archived": false, "description": null, "id": "manager1", "settings_str": "{}", "title": "Owner", "type": "multiple-person", "width": 80}, {"archived": false, "description": null, "id": "date4", "settings_str": "{}", "title": "Request date", "type": "date", "width": null}, {"archived": false, "description": null, "id": "status1", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"Evaluating\",\"1\":\"Done\",\"2\":\"Denied\",\"3\":\"Waiting for legal\",\"6\":\"Approved for POC\",\"11\":\"On hold\",\"14\":\"Waiting for vendor\",\"15\":\"Negotiation\",\"108\":\"Approved for use\"},\"labels_positions_v2\":{\"0\":0,\"1\":1,\"2\":7,\"3\":8,\"5\":9,\"6\":3,\"11\":6,\"14\":5,\"15\":4,\"108\":2},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"},\"3\":{\"color\":\"#0086c0\",\"border\":\"#3DB0DF\",\"var_name\":\"blue-links\"},\"6\":{\"color\":\"#037f4c\",\"border\":\"#006B38\",\"var_name\":\"grass-green\"},\"11\":{\"color\":\"#BB3354\",\"border\":\"#A42D4A\",\"var_name\":\"dark-red\"},\"14\":{\"color\":\"#784BD1\",\"border\":\"#8F4DC4\",\"var_name\":\"dark-purple\"},\"15\":{\"color\":\"#9CD326\",\"border\":\"#89B921\",\"var_name\":\"lime-green\"},\"108\":{\"color\":\"#4eccc6\",\"border\":\"#4eccc6\",\"var_name\":\"australia\"}}}", "title": "Procurement status", "type": "color", "width": null}, {"archived": false, "description": null, "id": "person", "settings_str": "{}", "title": "Manager", "type": "multiple-person", "width": 80}, {"archived": false, "description": null, "id": "status", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"On Hold\",\"1\":\"Approved\",\"2\":\"Declined\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "Manager approval", "type": "color", "width": null}, {"archived": false, "description": null, "id": "budget_owner", "settings_str": "{}", "title": "POC owner", "type": "multiple-person", "width": 80}, {"archived": false, "description": null, "id": "budget_owner_approval4", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"On Hold\",\"1\":\"Approved\",\"2\":\"Declined\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "POC status", "type": "color", "width": null}, {"archived": false, "description": null, "id": "manager", "settings_str": "{}", "title": "Budget owner", "type": "multiple-person", "width": 80}, {"archived": false, "description": null, "id": "status4", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"On Hold\",\"1\":\"Approved\",\"2\":\"Declined\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "Budget owner approval", "type": "color", "width": 185}, {"archived": false, "description": null, "id": "people", "settings_str": "{}", "title": "Procurement team", "type": "multiple-person", "width": null}, {"archived": false, "description": null, "id": "budget_owner_approval", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"On Hold\",\"1\":\"Approved\",\"2\":\"Declined\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "Procurement approval", "type": "color", "width": null}, {"archived": false, "description": null, "id": "procurement_team", "settings_str": "{}", "title": "Finance", "type": "multiple-person", "width": null}, {"archived": false, "description": null, "id": "procurement_approval", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"On Hold\",\"1\":\"Approved\",\"2\":\"Declined\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "Finance approval", "type": "color", "width": null}, {"archived": false, "description": null, "id": "finance", "settings_str": "{}", "title": "Legal", "type": "multiple-person", "width": null}, {"archived": false, "description": null, "id": "finance_approval", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"On Hold\",\"1\":\"Approved\",\"2\":\"Redlines\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "Legal approval", "type": "color", "width": null}, {"archived": false, "description": null, "id": "file", "settings_str": "{}", "title": "File", "type": "file", "width": null}, {"archived": false, "description": null, "id": "legal", "settings_str": "{}", "title": "Security", "type": "multiple-person", "width": null}, {"archived": false, "description": null, "id": "legal_approval", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"On Hold\",\"1\":\"Approved\",\"2\":\"Declined\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "Security approval", "type": "color", "width": null}, {"archived": false, "description": null, "id": "date", "settings_str": "{\"hide_footer\":false}", "title": "Renewal date", "type": "date", "width": null}, {"archived": false, "description": null, "id": "last_updated", "settings_str": "{}", "title": "Last updated", "type": "pulse-updated", "width": 129}], "communication": null, "description": "Many IT departments need to handle the procurement process for new services. The essence of this board is to streamline this process by providing an intuitive structure that supports collaboration and efficiency.", "groups": [{"archived": false, "color": "#579bfc", "deleted": false, "id": "topics", "position": "65536", "title": "Reviewing"}, {"archived": false, "color": "#FF642E", "deleted": false, "id": "new_group", "position": "98304.0", "title": "Corporate IT"}, {"archived": false, "color": "#037f4c", "deleted": false, "id": "new_group2816", "position": "114688.0", "title": "Finance"}], "id": "3555407826", "name": "Procurement process", "owners": [{"id": 36694549}], "creator": {"id": 36694549}, "permissions": "everyone", "pos": null, "state": "active", "subscribers": [{"id": 36694549}], "tags": [], "top_group": {"id": "topics"}, "updated_at": "2023-09-22T09:35:45Z", "updates": [], "views": [], "workspace": null, "updated_at_int": 1695375345}, "emitted_at": 1696447529797} +{"stream": "boards", "data": {"board_kind": "public", "type": "board", "columns": [{"archived": false, "description": null, "id": "name", "settings_str": "{}", "title": "Name", "type": "name", "width": 400}, {"archived": false, "description": null, "id": "person", "settings_str": "{}", "title": "Person", "type": "multiple-person", "width": null}, {"archived": false, "description": null, "id": "status", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"Working on it\",\"1\":\"Done\",\"2\":\"Stuck\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "Status", "type": "color", "width": null}, {"archived": false, "description": null, "id": "date4", "settings_str": "{}", "title": "Date", "type": "date", "width": null}, {"archived": false, "description": null, "id": "tags", "settings_str": "{\"hide_footer\":false}", "title": "Tags", "type": "tag", "width": null}], "communication": null, "description": null, "groups": [{"archived": false, "color": "#579bfc", "deleted": false, "id": "topics", "position": "65536", "title": "Group Title"}, {"archived": false, "color": "#a25ddc", "deleted": false, "id": "group_title", "position": "98304", "title": "Group Title"}, {"archived": false, "color": "#808080", "deleted": false, "id": "new_group", "position": "163840.0", "title": "New Group unit board"}], "id": "4635211873", "name": "New Board", "owners": [{"id": 36694549}], "creator": {"id": 36694549}, "permissions": "everyone", "pos": null, "state": "active", "subscribers": [{"id": 36694549}], "tags": [], "top_group": {"id": "topics"}, "updated_at": "2023-06-20T12:12:46Z", "updates": [{"id": "2223820299"}, {"id": "2223818363"}], "views": [], "workspace": {"id": 2845647, "name": "Test workspace", "kind": "open", "description": null}, "updated_at_int": 1687263166}, "emitted_at": 1702496562635} +{"stream": "boards", "data": {"board_kind": "public", "type": "board", "columns": [{"archived": false, "description": null, "id": "name", "settings_str": "{}", "title": "Name", "type": "name", "width": 380}, {"archived": false, "description": null, "id": "manager1", "settings_str": "{}", "title": "Owner", "type": "multiple-person", "width": 80}, {"archived": false, "description": null, "id": "date4", "settings_str": "{}", "title": "Request date", "type": "date", "width": null}, {"archived": false, "description": null, "id": "status1", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"Evaluating\",\"1\":\"Done\",\"2\":\"Denied\",\"3\":\"Waiting for legal\",\"6\":\"Approved for POC\",\"11\":\"On hold\",\"14\":\"Waiting for vendor\",\"15\":\"Negotiation\",\"108\":\"Approved for use\"},\"labels_positions_v2\":{\"0\":0,\"1\":1,\"2\":7,\"3\":8,\"5\":9,\"6\":3,\"11\":6,\"14\":5,\"15\":4,\"108\":2},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"},\"3\":{\"color\":\"#0086c0\",\"border\":\"#3DB0DF\",\"var_name\":\"blue-links\"},\"6\":{\"color\":\"#037f4c\",\"border\":\"#006B38\",\"var_name\":\"grass-green\"},\"11\":{\"color\":\"#BB3354\",\"border\":\"#A42D4A\",\"var_name\":\"dark-red\"},\"14\":{\"color\":\"#784BD1\",\"border\":\"#8F4DC4\",\"var_name\":\"dark-purple\"},\"15\":{\"color\":\"#9CD326\",\"border\":\"#89B921\",\"var_name\":\"lime-green\"},\"108\":{\"color\":\"#4eccc6\",\"border\":\"#4eccc6\",\"var_name\":\"australia\"}}}", "title": "Procurement status", "type": "color", "width": null}, {"archived": false, "description": null, "id": "person", "settings_str": "{}", "title": "Manager", "type": "multiple-person", "width": 80}, {"archived": false, "description": null, "id": "status", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"On Hold\",\"1\":\"Approved\",\"2\":\"Declined\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "Manager approval", "type": "color", "width": null}, {"archived": false, "description": null, "id": "budget_owner", "settings_str": "{}", "title": "POC owner", "type": "multiple-person", "width": 80}, {"archived": false, "description": null, "id": "budget_owner_approval4", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"On Hold\",\"1\":\"Approved\",\"2\":\"Declined\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "POC status", "type": "color", "width": null}, {"archived": false, "description": null, "id": "manager", "settings_str": "{}", "title": "Budget owner", "type": "multiple-person", "width": 80}, {"archived": false, "description": null, "id": "status4", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"On Hold\",\"1\":\"Approved\",\"2\":\"Declined\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "Budget owner approval", "type": "color", "width": 185}, {"archived": false, "description": null, "id": "people", "settings_str": "{}", "title": "Procurement team", "type": "multiple-person", "width": null}, {"archived": false, "description": null, "id": "budget_owner_approval", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"On Hold\",\"1\":\"Approved\",\"2\":\"Declined\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "Procurement approval", "type": "color", "width": null}, {"archived": false, "description": null, "id": "procurement_team", "settings_str": "{}", "title": "Finance", "type": "multiple-person", "width": null}, {"archived": false, "description": null, "id": "procurement_approval", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"On Hold\",\"1\":\"Approved\",\"2\":\"Declined\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "Finance approval", "type": "color", "width": null}, {"archived": false, "description": null, "id": "finance", "settings_str": "{}", "title": "Legal", "type": "multiple-person", "width": null}, {"archived": false, "description": null, "id": "finance_approval", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"On Hold\",\"1\":\"Approved\",\"2\":\"Redlines\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "Legal approval", "type": "color", "width": null}, {"archived": false, "description": null, "id": "file", "settings_str": "{}", "title": "File", "type": "file", "width": null}, {"archived": false, "description": null, "id": "legal", "settings_str": "{}", "title": "Security", "type": "multiple-person", "width": null}, {"archived": false, "description": null, "id": "legal_approval", "settings_str": "{\"done_colors\":[1],\"labels\":{\"0\":\"On Hold\",\"1\":\"Approved\",\"2\":\"Declined\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "Security approval", "type": "color", "width": null}, {"archived": false, "description": null, "id": "date", "settings_str": "{\"hide_footer\":false}", "title": "Renewal date", "type": "date", "width": null}, {"archived": false, "description": null, "id": "last_updated", "settings_str": "{}", "title": "Last updated", "type": "pulse-updated", "width": 129}], "communication": null, "description": "Many IT departments need to handle the procurement process for new services. The essence of this board is to streamline this process by providing an intuitive structure that supports collaboration and efficiency.", "groups": [{"archived": false, "color": "#579bfc", "deleted": false, "id": "topics", "position": "65536", "title": "Reviewing"}, {"archived": false, "color": "#FF642E", "deleted": false, "id": "new_group", "position": "98304.0", "title": "Corporate IT"}, {"archived": false, "color": "#037f4c", "deleted": false, "id": "new_group2816", "position": "114688.0", "title": "Finance"}], "id": "3555407826", "name": "Procurement process", "owners": [{"id": 36694549}], "creator": {"id": 36694549}, "permissions": "everyone", "pos": null, "state": "active", "subscribers": [{"id": 36694549}], "tags": [], "top_group": {"id": "topics"}, "updated_at": "2022-11-21T14:36:50Z", "updates": [], "views": [], "workspace": null, "updated_at_int": 1669041410}, "emitted_at": 1702496562643} +{"stream": "boards", "data": {"board_kind": "public", "type": "board", "columns": [{"archived": false, "description": null, "id": "name", "settings_str": "{}", "title": "Name", "type": "name", "width": 523}, {"archived": false, "description": null, "id": "text4", "settings_str": "{}", "title": "SN", "type": "text", "width": null}, {"archived": false, "description": null, "id": "status", "settings_str": "{\"done_colors\":[1],\"hide_footer\":true,\"labels\":{\"0\":\"Out for repair\",\"1\":\"Working well\",\"2\":\"Needs replacement\"},\"labels_positions_v2\":{\"0\":0,\"1\":2,\"2\":1,\"5\":3},\"labels_colors\":{\"0\":{\"color\":\"#fdab3d\",\"border\":\"#E99729\",\"var_name\":\"orange\"},\"1\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"2\":{\"color\":\"#e2445c\",\"border\":\"#CE3048\",\"var_name\":\"red-shadow\"}}}", "title": "Status", "type": "color", "width": null}, {"archived": false, "description": null, "id": "date4", "settings_str": "{}", "title": "Date given to current owner", "type": "date", "width": 204}, {"archived": false, "description": null, "id": "text", "settings_str": "{}", "title": "Current owner", "type": "text", "width": null}, {"archived": false, "description": null, "id": "date_given_to_current_owner", "settings_str": "{}", "title": "Last checked", "type": "date", "width": 129}], "communication": null, "description": "Welcome to your inventory management board. This is the place to track and manage all of your IT equipment inventory.", "groups": [{"archived": false, "color": "#BB3354", "deleted": false, "id": "duplicate_of_tvs___projectors", "position": "65408", "title": "Out of service"}, {"archived": false, "color": "#579bfc", "deleted": false, "id": "topics", "position": "65536", "title": "Laptops"}, {"archived": false, "color": "#a25ddc", "deleted": false, "id": "group_title", "position": "98304", "title": "Monitors"}, {"archived": false, "color": "#037f4c", "deleted": false, "id": "new_group", "position": "163840.0", "title": "TVs & projectors"}], "id": "3555407785", "name": "Inventory management", "owners": [{"id": 36694549}], "creator": {"id": 36694549}, "permissions": "everyone", "pos": null, "state": "active", "subscribers": [{"id": 36694549}], "tags": [], "top_group": {"id": "duplicate_of_tvs___projectors"}, "updated_at": "2022-11-21T14:36:49Z", "updates": [], "views": [], "workspace": null, "updated_at_int": 1669041409}, "emitted_at": 1702496562649} {"stream": "tags", "data": {"color": "#00c875", "id": 19038090, "name": "open"}, "emitted_at": 1690884065804} {"stream": "tags", "data": {"color": "#fdab3d", "id": 19038091, "name": "closed"}, "emitted_at": 1690884065806} {"stream": "updates", "data": {"assets": [{"created_at": "2023-06-15T16:19:31Z", "file_extension": ".jpg", "file_size": 116107, "id": "919077184", "name": "black_cat.jpg", "original_geometry": "473x600", "public_url": "https://files-monday-com.s3.amazonaws.com/14202902/resources/919077184/black_cat.jpg?response-content-disposition=attachment&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA4MPVJMFXGWGLJTLY%2F20230801%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230801T100107Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=5d2d3ca95375589e620f89630d58ff0f7417f1ddd8968ceb57af854657718564", "uploaded_by": {"id": 36694549}, "url": "https://airbyte-unit.monday.com/protected_static/14202902/resources/919077184/black_cat.jpg", "url_thumbnail": "https://airbyte-unit.monday.com/protected_static/14202902/resources/919077184/thumb_small-black_cat.jpg"}], "body": "", "created_at": "2023-06-15T16:19:36Z", "creator_id": "36694549", "id": "2223820299", "item_id": "4635211945", "replies": [], "text_body": "", "updated_at": "2023-06-15T16:19:36Z"}, "emitted_at": 1690884067025} {"stream": "updates", "data": {"assets": [], "body": "



", "created_at": "2023-06-15T16:18:50Z", "creator_id": "36694549", "id": "2223818363", "item_id": "4635211945", "replies": [], "text_body": "", "updated_at": "2023-06-15T16:18:50Z"}, "emitted_at": 1690884067027} {"stream": "updates", "data": {"assets": [], "body": "

\ufeffTest

", "created_at": "2022-11-21T14:41:21Z", "creator_id": "36694549", "id": "1825302913", "item_id": "3555437747", "replies": [{"id": "1825303266", "creator_id": "36694549", "created_at": "2022-11-21T14:41:29Z", "text_body": "Test test", "updated_at": "2022-11-21T14:41:29Z", "body": "

\ufeffTest test

"}, {"id": "2223806079", "creator_id": "36694549", "created_at": "2023-06-15T16:14:13Z", "text_body": "", "updated_at": "2023-06-15T16:14:13Z", "body": "



"}], "text_body": "Test", "updated_at": "2023-06-15T16:14:13Z"}, "emitted_at": 1690884067029} -{"stream": "users", "data": {"birthday": null, "country_code": "UA", "created_at": "2022-11-21T14:03:00Z", "join_date": null, "email": "integration-test@airbyte.io", "enabled": true, "id": 36694549, "is_admin": true, "is_guest": false, "is_pending": false, "is_view_only": false, "is_verified": true, "location": null, "mobile_phone": null, "name": "Airbyte Team", "phone": "", "photo_original": "https://files.monday.com/use1/photos/36694549/original/36694549-user_photo_2022_11_21_14_10_42.png?1669039842", "photo_small": "https://files.monday.com/use1/photos/36694549/small/36694549-user_photo_2022_11_21_14_10_42.png?1669039842", "photo_thumb": "https://files.monday.com/use1/photos/36694549/thumb/36694549-user_photo_2022_11_21_14_10_42.png?1669039842", "photo_thumb_small": "https://files.monday.com/use1/photos/36694549/thumb_small/36694549-user_photo_2022_11_21_14_10_42.png?1669039842", "photo_tiny": "https://files.monday.com/use1/photos/36694549/tiny/36694549-user_photo_2022_11_21_14_10_42.png?1669039842", "time_zone_identifier": "Europe/Kiev", "title": "Airbyte Developer Account", "url": "https://airbyte-unit.monday.com/users/36694549", "utc_hours_diff": 3}, "emitted_at": 1690884067354} -{"stream": "users", "data": {"birthday": null, "country_code": "UA", "created_at": "2022-11-21T14:33:18Z", "join_date": null, "email": "iryna.grankova@airbyte.io", "enabled": true, "id": 36695702, "is_admin": false, "is_guest": false, "is_pending": false, "is_view_only": false, "is_verified": true, "location": null, "mobile_phone": null, "name": "Iryna Grankova", "phone": null, "photo_original": "https://files.monday.com/use1/photos/36695702/original/36695702-user_photo_initials_2022_11_21_14_34_12.png?1669041252", "photo_small": "https://files.monday.com/use1/photos/36695702/small/36695702-user_photo_initials_2022_11_21_14_34_12.png?1669041252", "photo_thumb": "https://files.monday.com/use1/photos/36695702/thumb/36695702-user_photo_initials_2022_11_21_14_34_12.png?1669041252", "photo_thumb_small": "https://files.monday.com/use1/photos/36695702/thumb_small/36695702-user_photo_initials_2022_11_21_14_34_12.png?1669041252", "photo_tiny": "https://files.monday.com/use1/photos/36695702/tiny/36695702-user_photo_initials_2022_11_21_14_34_12.png?1669041252", "time_zone_identifier": "Europe/Athens", "title": null, "url": "https://airbyte-unit.monday.com/users/36695702", "utc_hours_diff": 3}, "emitted_at": 1690884067356} +{"stream": "users", "data": {"birthday": null, "country_code": "UA", "created_at": "2022-11-21T14:03:00Z", "join_date": null, "email": "integration-test@airbyte.io", "enabled": true, "id": 36694549, "is_admin": true, "is_guest": false, "is_pending": false, "is_view_only": false, "is_verified": true, "location": null, "mobile_phone": null, "name": "Airbyte Team", "phone": "", "photo_original": "https://files.monday.com/use1/photos/36694549/original/36694549-user_photo_2022_11_21_14_10_42.png?1669039842", "photo_small": "https://files.monday.com/use1/photos/36694549/small/36694549-user_photo_2022_11_21_14_10_42.png?1669039842", "photo_thumb": "https://files.monday.com/use1/photos/36694549/thumb/36694549-user_photo_2022_11_21_14_10_42.png?1669039842", "photo_thumb_small": "https://files.monday.com/use1/photos/36694549/thumb_small/36694549-user_photo_2022_11_21_14_10_42.png?1669039842", "photo_tiny": "https://files.monday.com/use1/photos/36694549/tiny/36694549-user_photo_2022_11_21_14_10_42.png?1669039842", "time_zone_identifier": "Europe/Kiev", "title": "Airbyte Developer Account", "url": "https://airbyte-unit.monday.com/users/36694549", "utc_hours_diff": 2}, "emitted_at": 1702496564648} +{"stream": "users", "data": {"birthday": null, "country_code": "UA", "created_at": "2022-11-21T14:33:18Z", "join_date": null, "email": "iryna.grankova@airbyte.io", "enabled": true, "id": 36695702, "is_admin": false, "is_guest": false, "is_pending": false, "is_view_only": false, "is_verified": true, "location": null, "mobile_phone": null, "name": "Iryna Grankova", "phone": null, "photo_original": "https://files.monday.com/use1/photos/36695702/original/36695702-user_photo_initials_2022_11_21_14_34_12.png?1669041252", "photo_small": "https://files.monday.com/use1/photos/36695702/small/36695702-user_photo_initials_2022_11_21_14_34_12.png?1669041252", "photo_thumb": "https://files.monday.com/use1/photos/36695702/thumb/36695702-user_photo_initials_2022_11_21_14_34_12.png?1669041252", "photo_thumb_small": "https://files.monday.com/use1/photos/36695702/thumb_small/36695702-user_photo_initials_2022_11_21_14_34_12.png?1669041252", "photo_tiny": "https://files.monday.com/use1/photos/36695702/tiny/36695702-user_photo_initials_2022_11_21_14_34_12.png?1669041252", "time_zone_identifier": "Europe/Athens", "title": null, "url": "https://airbyte-unit.monday.com/users/36695702", "utc_hours_diff": 2}, "emitted_at": 1702496564650} {"stream": "workspaces", "data": {"created_at": "2023-06-08T11:26:44Z", "description": null, "id": 2845647, "kind": "open", "name": "Test workspace", "state": "active", "account_product": {"id": 2248222, "kind": "core"}, "owners_subscribers": [{"id": 36694549}], "settings": {"icon": {"color": "#FDAB3D", "image": null}}, "team_owners_subscribers": [], "teams_subscribers": [], "users_subscribers": [{"id": 36694549}]}, "emitted_at": 1690884067856} {"stream": "activity_logs", "data": {"id": "81d07d4d-414d-458e-b44c-fef36e44c424", "event": "create_pulse", "data": "{\"board_id\":4635211873,\"group_id\":\"new_group\",\"group_name\":\"New Group unit board\",\"group_color\":\"#808080\",\"is_top_group\":false,\"pulse_id\":4672924165,\"pulse_name\":\"Item 7\",\"column_values_json\":\"{}\"}", "entity": "pulse", "created_at": "16872631837419768", "created_at_int": 1687263183, "pulse_id": 4672924165}, "emitted_at": 1690884068262} {"stream": "activity_logs", "data": {"id": "c0aa4bab-d3a4-4f13-8942-934178c0238a", "event": "update_column_value", "data": "{\"board_id\":4635211873,\"group_id\":\"group_title\",\"is_top_group\":false,\"pulse_id\":4672922929,\"pulse_name\":\"Item 6\",\"column_id\":\"status\",\"column_type\":\"color\",\"column_title\":\"Status\",\"value\":{\"label\":{\"index\":1,\"text\":\"Done\",\"style\":{\"color\":\"#00c875\",\"border\":\"#00B461\",\"var_name\":\"green-shadow\"},\"is_done\":true},\"post_id\":null},\"previous_value\":null,\"is_column_with_hide_permissions\":false}", "entity": "pulse", "created_at": "16872631743009674", "created_at_int": 1687263174, "pulse_id": 4672922929}, "emitted_at": 1690884068266} diff --git a/airbyte-integrations/connectors/source-monday/metadata.yaml b/airbyte-integrations/connectors/source-monday/metadata.yaml index ac999afd827d8..3ec648c41fe19 100644 --- a/airbyte-integrations/connectors/source-monday/metadata.yaml +++ b/airbyte-integrations/connectors/source-monday/metadata.yaml @@ -1,12 +1,18 @@ data: + ab_internal: + ql: 200 + sl: 200 allowedHosts: hosts: - api.monday.com + connectorBuildOptions: + baseImage: docker.io/airbyte/python-connector-base:1.2.0@sha256:c22a9d97464b69d6ef01898edf3f8612dc11614f05a84984451dde195f337db9 connectorSubtype: api connectorType: source definitionId: 80a54ea2-9959-4040-aac1-eee42423ec9b - dockerImageTag: 1.1.3 + dockerImageTag: 1.1.4 dockerRepository: airbyte/source-monday + documentationUrl: https://docs.airbyte.com/integrations/sources/monday githubIssueLabel: source-monday icon: monday.svg license: MIT @@ -17,12 +23,8 @@ data: oss: enabled: true releaseStage: generally_available - documentationUrl: https://docs.airbyte.com/integrations/sources/monday + supportLevel: certified tags: - language:low-code - language:python - ab_internal: - sl: 200 - ql: 400 - supportLevel: certified metadataSpecVersion: "1.0" diff --git a/airbyte-integrations/connectors/source-monday/unit_tests/test_components.py b/airbyte-integrations/connectors/source-monday/unit_tests/test_components.py index 49be96073f1e1..670aff5e4e449 100644 --- a/airbyte-integrations/connectors/source-monday/unit_tests/test_components.py +++ b/airbyte-integrations/connectors/source-monday/unit_tests/test_components.py @@ -7,6 +7,7 @@ from unittest.mock import MagicMock, Mock import pytest +from airbyte_cdk.models import AirbyteMessage, SyncMode, Type from airbyte_cdk.sources.declarative.partition_routers.substream_partition_router import ParentStreamConfig from airbyte_cdk.sources.streams import Stream from requests import Response @@ -99,3 +100,87 @@ def test_null_records(caplog): {"board_kind": "private", "id": "1234566", "updated_at": "2023-08-15T10:30:54Z", "updated_at_int": 1692095454}, ] assert records == expected_records + + +@pytest.fixture +def mock_parent_stream(): + + def mock_parent_stream_slices(*args, **kwargs): + return iter([{"ids": [123]}]) + + mock_stream = MagicMock(spec=Stream) + mock_stream.primary_key = "id" # Example primary key + mock_stream.stream_slices = mock_parent_stream_slices + mock_stream.parent_config = ParentStreamConfig( + stream=mock_stream, + parent_key="id", + partition_field="parent_stream_id", + parameters={}, + config={}, + ) + + return mock_stream + +@pytest.mark.parametrize("stream_state, parent_records, expected_slices", + [ + ({}, [], [{}]), + ( + {"updated_at": "2022-01-01T00:00:00Z"}, + [AirbyteMessage( + type=Type.RECORD, + record={ "data": {"id": 123, "name": "Sample Record", "updated_at": "2023-01-01T00:00:00Z"}, "stream": "projects", "emitted_at": 1632095449} + )], + [{'parent_stream_id': [123]}] + ), + ( + {"updated_at": "2022-01-01T00:00:00Z"}, + AirbyteMessage(type=Type.LOG), + [] + ) + ], + ids=[ + "no stream state", + "successfully read parent record", + "skip non_record AirbyteMessage" + ] +) +def test_read_parent_stream(mock_parent_stream, stream_state, parent_records, expected_slices): + + slicer = IncrementalSubstreamSlicer( + config={}, + parameters={}, + cursor_field="updated_at", + parent_stream_configs=[mock_parent_stream.parent_config], + nested_items_per_page=10 + ) + + mock_parent_stream.read_records = MagicMock(return_value=parent_records) + slicer.parent_cursor_field = "updated_at" + + slices = list(slicer.read_parent_stream( + sync_mode=SyncMode.full_refresh, + cursor_field="updated_at", + stream_state=stream_state + )) + + assert slices == expected_slices + + +def test_set_initial_state(): + + slicer = IncrementalSubstreamSlicer( + config={}, + parameters={}, + cursor_field="updated_at_int", + parent_stream_configs=[MagicMock(parent_stream_name="parent_stream")], + nested_items_per_page=10 + ) + + initial_stream_state = { + "updated_at_int": 1662459010, + "parent_stream": {"parent_cursor_field": 1662459011} + } + + expected_state = { "updated_at_int": 1662459010 } + slicer.set_initial_state(initial_stream_state) + assert slicer._state == expected_state diff --git a/airbyte-integrations/connectors/source-monday/unit_tests/test_extractor.py b/airbyte-integrations/connectors/source-monday/unit_tests/test_extractor.py new file mode 100644 index 0000000000000..cf6a4bfc871ce --- /dev/null +++ b/airbyte-integrations/connectors/source-monday/unit_tests/test_extractor.py @@ -0,0 +1,36 @@ +# +# Copyright (c) 2023 Airbyte, Inc., all rights reserved. +# + +from unittest.mock import MagicMock + +from source_monday.extractor import MondayActivityExtractor + + +def test_extract_records(): + # Mock the response + response = MagicMock() + response_body = { + "data": { + "boards": [ + { + "activity_logs": [ + { + "data": "{\"pulse_id\": 123}", + "entity": "pulse", + "created_at": "16367386880000000" + } + ] + } + ] + } + } + + response.json.return_value = response_body + extractor = MondayActivityExtractor(parameters={}) + records = extractor.extract_records(response) + + # Assertions + assert len(records) == 1 + assert records[0]["pulse_id"] == 123 + assert records[0]["created_at_int"] == 1636738688 diff --git a/airbyte-integrations/connectors/source-monday/unit_tests/test_graphql_requester.py b/airbyte-integrations/connectors/source-monday/unit_tests/test_graphql_requester.py index 2658134d6cb58..f080b8c085b49 100644 --- a/airbyte-integrations/connectors/source-monday/unit_tests/test_graphql_requester.py +++ b/airbyte-integrations/connectors/source-monday/unit_tests/test_graphql_requester.py @@ -5,7 +5,9 @@ from unittest.mock import MagicMock import pytest +from airbyte_cdk.sources.declarative.interpolation.interpolated_string import InterpolatedString from airbyte_cdk.sources.declarative.requesters.requester import HttpMethod +from airbyte_cdk.sources.declarative.schema.json_file_schema_loader import JsonFileSchemaLoader from source_monday import MondayGraphqlRequester nested_object_schema = { @@ -89,3 +91,62 @@ def test_get_request_params(mocker, input_schema, graphql_query, stream_name, co config=config, ) assert requester.get_request_params(stream_state={}, stream_slice={}, next_page_token=next_page_token) == graphql_query + + +@pytest.fixture +def monday_requester(): + return MondayGraphqlRequester( + name="a name", + url_base="https://api.monday.com/v2", + path="a-path", + config={}, + parameters={"name": "activity_logs"}, + limit=InterpolatedString.create("100", parameters={"name": "activity_logs"}), + nested_limit=InterpolatedString.create("100", parameters={"name": "activity_logs"}), + ) + +def test_get_schema_root_properties(mocker, monday_requester): + mock_schema = { + "properties": { + "updated_at_int": {"type": "integer"}, + "created_at_int": {"type": "integer"}, + "pulse_id": {"type": "integer"}, + "board_id": {"type": "integer"}, + "other_field": {"type": "string"}, + "yet_another_field": {"type": "boolean"} + } + } + + mocker.patch.object(JsonFileSchemaLoader, 'get_json_schema', return_value=mock_schema) + requester = monday_requester + result_schema = requester._get_schema_root_properties() + + assert result_schema == { + "other_field": { "type": "string" }, + "yet_another_field": { "type": "boolean" } + } + + +def test_build_activity_query(mocker, monday_requester): + + mock_stream_state = { "updated_at_int": 1636738688 } + object_arguments = { "stream_state": mock_stream_state } + mocker.patch.object(MondayGraphqlRequester, '_get_object_arguments', return_value="stream_state:{{ stream_state['updated_at_int'] }}") + requester = monday_requester + + result = requester._build_activity_query(object_name="activity_logs", field_schema={}, sub_page=None, **object_arguments) + assert result == "boards(stream_state:{{ stream_state['updated_at_int'] }}){activity_logs(stream_state:{{ stream_state['updated_at_int'] }}){}}" + + +def test_build_items_incremental_query(monday_requester): + + object_name = "test_items" + field_schema = { + "id": {"type": "integer"}, + "name": {"type": "string"}, + } + stream_slice = {"ids": [1, 2, 3]} + + built_query = monday_requester._build_items_incremental_query(object_name, field_schema, stream_slice) + + assert built_query == 'items(limit:100,ids:[1, 2, 3]){id,name}' diff --git a/docs/integrations/sources/monday.md b/docs/integrations/sources/monday.md index 4f0310162d75d..596d6ae1c6867 100644 --- a/docs/integrations/sources/monday.md +++ b/docs/integrations/sources/monday.md @@ -1,5 +1,7 @@ # Monday +This page contains the setup guide and reference information for the [Monday](https://monday.com/) source connector. + ## Prerequisites * Monday API Token / Monday Access Token @@ -15,18 +17,20 @@ You can get the API token for Monday by going to Profile picture (bottom left co 3. On the Set up the source page, enter the name for the Monday connector and select **Monday** from the Source type dropdown. 4. Fill in your API Key or authenticate using OAuth and then click **Set up source**. -### Connect using `OAuth 2.0` option: +### Connect using `OAuth 2.0` option + 1. Select `OAuth2.0` in `Authorization Method`. 2. Click on `authenticate your Monday account`. -2. Proceed the authentication using your credentials for your Monday account. +3. Proceed with the authentication using the credentials for your Monday account. + +### Connect using `API Token` option -### Connect using `API Token` option: 1. Generate an API Token as described [here](https://developer.monday.com/api-reference/docs/authentication). 2. Use the generated `api_token` in the Airbyte connection. ## Supported sync modes -The Monday supports full refresh syncs +The Monday source connector supports the following features: | Feature | Supported? | |:------------------|:-----------| @@ -49,6 +53,7 @@ Several output streams are available from this source: * [Workspaces](https://developer.monday.com/api-reference/docs/workspaces) Important Notes: + * `Columns` are available from the `Boards` stream. By syncing the `Boards` stream you will get the `Columns` for each `Board` synced in the database The typical name of the table depends on the `destination` you use like `boards.columns`, for instance. @@ -61,16 +66,15 @@ Ids of boards and items are extracted from activity logs events and used to sele Some data may be lost if the time between incremental syncs is longer than the activity logs retention time for your plan. Check your Monday plan at https://monday.com/pricing. - ## Performance considerations The Monday connector should not run into Monday API limitations under normal usage. Please [create an issue](https://github.com/airbytehq/airbyte/issues) if you see any rate limit issues that are not automatically retried successfully. - ## Changelog | Version | Date | Pull Request | Subject | |:--------|:-----------|:----------------------------------------------------------|:------------------------------------------------------------------------| +| 1.1.4 | 2023-12-13 | [33448](https://github.com/airbytehq/airbyte/pull/33448) | Increase test coverage and migrate to base image | | 1.1.3 | 2023-09-23 | [30248](https://github.com/airbytehq/airbyte/pull/30248) | Add new field "type" to board stream | | 1.1.2 | 2023-08-23 | [29777](https://github.com/airbytehq/airbyte/pull/29777) | Add retry for `502` error | | 1.1.1 | 2023-08-15 | [29429](https://github.com/airbytehq/airbyte/pull/29429) | Ignore `null` records in response |