Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎉 Source Looker: Migrate Looker source to CDK structure, add SAT testing #7284

Merged
merged 5 commits into from
Oct 27, 2021

Conversation

yevhenii-ldv
Copy link
Contributor

What

resolves #7246 .

How

Describe the solution

Recommended reading order

  1. x.java
  2. y.python

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Community member or Airbyter

  • Grant edit access to maintainers (instructions)
  • Secrets in the connector's spec are annotated with airbyte_secret
  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
  • PR name follows PR naming conventions
  • Connector version bumped like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • Credentials added to Github CI. Instructions.
  • /test connector=connectors/<name> command is passing.
  • New Connector version released on Dockerhub by running the /publish command described here

@github-actions github-actions bot added area/connectors Connector related issues area/documentation Improvements or additions to documentation labels Oct 22, 2021
@yevhenii-ldv
Copy link
Contributor Author

yevhenii-ldv commented Oct 22, 2021

/test connector=connectors/source-looker

🕑 connectors/source-looker https://github.com/airbytehq/airbyte/actions/runs/1372927592
✅ connectors/source-looker https://github.com/airbytehq/airbyte/actions/runs/1372927592
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  860    419    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                        Stmts   Miss  Cover
	 -----------------------------------------------
	 source_looker/__init__.py       2      2     0%
	 source_looker/client.py       224    224     0%
	 source_looker/source.py         4      4     0%
	 unit_tests/unit_test.py::test_example_method �[32mPASSED�[0m/actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-looker/.venv/lib/python3.8/site-packages/coverage/control.py:761: CoverageWarning: No data was collected. (no-data-collected)
	 -----------------------------------------------
	 TOTAL                         230    230     0%

@jrhizor jrhizor temporarily deployed to more-secrets October 22, 2021 16:07 Inactive
Comment on lines +18 to +25
[
"homepages",
"looks",
"run_looks",
"scheduled_plans",
"user_attribute_group_values",
"user_login_lockouts",
"user_sessions",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have an issue to fill those streams?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Several of these streams are dynamic. But in the scope of this task, I did not fill the account with data.

dependencies {
implementation files(project(':airbyte-integrations:bases:base-standard-source-test-file').airbyteDocker.outputs)
implementation files(project(':airbyte-integrations:bases:base-python').airbyteDocker.outputs)
implementation files(project(':airbyte-integrations:bases:source-acceptance-test').airbyteDocker.outputs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't really need this dependency cause we are using SAT image from dockerhub

@@ -22,7 +22,7 @@
}
},
"type": ["null", "object"],
"additionalProperties": false
"additionalProperties": true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just remove additionalProperties, its true by default.

},
"type": ["null", "array"]
},
"external_id": {
"type": ["null", "string"]
},
"id": {
"type": ["null", "string"]
"type": ["null", "string", "integer"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we leave it as string and enbale default schema type transformation ? It looks like primary key and its better to have one type here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This connector is custom written and did not use the Stream class from the Airbyte CDK, so unfortunately we cannot apply this approach to it.

@yevhenii-ldv
Copy link
Contributor Author

yevhenii-ldv commented Oct 25, 2021

/test connector=connectors/source-looker

🕑 connectors/source-looker https://github.com/airbytehq/airbyte/actions/runs/1381462525
✅ connectors/source-looker https://github.com/airbytehq/airbyte/actions/runs/1381462525
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  860    419    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                        Stmts   Miss  Cover
	 -----------------------------------------------
	 source_looker/__init__.py       2      2     0%
	 source_looker/client.py       224    224     0%
	 source_looker/source.py         4      4     0%
	 -----------------------------------------------
	 TOTAL                         230    230     0%

@yevhenii-ldv yevhenii-ldv temporarily deployed to more-secrets October 25, 2021 14:13 Inactive
@jrhizor jrhizor temporarily deployed to more-secrets October 25, 2021 14:16 Inactive
@yevhenii-ldv
Copy link
Contributor Author

yevhenii-ldv commented Oct 27, 2021

/test connector=connectors/source-looker

🕑 connectors/source-looker https://github.com/airbytehq/airbyte/actions/runs/1390566030
✅ connectors/source-looker https://github.com/airbytehq/airbyte/actions/runs/1390566030
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  860    419    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                        Stmts   Miss  Cover
	 -----------------------------------------------
	 source_looker/__init__.py       2      2     0%
	 source_looker/client.py       224    224     0%
	 source_looker/source.py         4      4     0%
	 -----------------------------------------------
	 TOTAL                         230    230     0%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------/actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-looker/.venv/lib/python3.8/site-packages/coverage/control.py:761: CoverageWarning: No data was collected. (no-data-collected)
	   self._warn("No data was collected.", slug="no-data-collected")

> Task :airbyte-integrations:connectors:source-looker:airbytePythonTest
> Task :airbyte-integrations:connectors:source-looker:assemble

> Task :airbyte-integrations:bases:source-acceptance-test:airbyteDocker
Successfully installed Deprecated-1.2.13 PyYAML-5.4.1 airbyte-cdk-0.1.32 attrs-21.2.0 backoff-1.11.1 certifi-2021.10.8 charset-normalizer-2.0.7 docker-4.4.4 dpath-2.0.5 fancycompleter-0.9.1 icdiff-1.9.1 idna-3.3 importlib-metadata-4.8.1 inflection-0.5.1 iniconfig-1.1.1 jsonref-0.2 jsonschema-3.2.0 multidict-5.2.0 packaging-21.0 pdbpp-0.10.3 pendulum-2.1.2 pluggy-1.0.0 pprintpp-0.4.0 py-1.10.0 pydantic-1.8.2 pygments-2.10.0 pyparsing-3.0.2 pyrepl-0.9.0 pyrsistent-0.18.0 pytest-6.2.5 pytest-sugar-0.9.4 pytest-timeout-1.4.2 python-dateutil-2.8.2 pytzdata-2020.1 requests-2.26.0 six-1.16.0 source-acceptance-test-0.0.0 termcolor-1.1.0 toml-0.10.2 typing-extensions-3.10.0.2 urllib3-1.26.7 vcrpy-4.1.1 websocket-client-1.2.1 wmctrl-0.4 wrapt-1.13.2 yarl-1.7.0 zipp-3.6.0
�[91mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
�[0m�[91mWARNING: You are using pip version 21.1.3; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
�[0mRemoving intermediate container 1cc56a1a5844
 ---> 0f78125acf7d
Step 9/11 : LABEL io.airbyte.version=0.1.24
 ---> Running in 24d2ac969ea0
Removing intermediate container 24d2ac969ea0
 ---> 58b59b7e6407
Step 10/11 : LABEL io.airbyte.name=airbyte/source-acceptance-test
 ---> Running in 2d4f933a68c4
Removing intermediate container 2d4f933a68c4
 ---> b4cbd49a248b
Step 11/11 : ENTRYPOINT ["python", "-m", "pytest", "-p", "source_acceptance_test.plugin"]
 ---> Running in 7d2877b71f71
Removing intermediate container 7d2877b71f71
 ---> 270ef2cf9f01
[Warning] One or more build-args [JDK_VERSION] were not consumed
Successfully built 270ef2cf9f01
Successfully tagged airbyte/source-acceptance-test:dev

> Task :airbyte-integrations:connectors:source-looker:airbyteDocker
Execution optimizations have been disabled for task ':airbyte-integrations:connectors:source-looker:airbyteDocker' to ensure correctness due to the following reasons:
  - Gradle detected a problem with the following location: '/actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-looker'. Reason: Task ':airbyte-integrations:connectors:source-looker:airbyteDocker' uses this output of task ':airbyte-integrations:connectors:source-looker:compileTestJava' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.2/userguide/validation_problems.html#implicit_dependency for more details about this problem.
  - Gradle detected a problem with the following location: '/actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-looker'. Reason: Task ':airbyte-integrations:connectors:source-looker:airbyteDocker' uses this output of task ':airbyte-integrations:connectors:source-looker:processTestResources' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.2/userguide/validation_problems.html#implicit_dependency for more details about this problem.
  - Gradle detected a problem with the following location: '/actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-looker'. Reason: Task ':airbyte-integrations:connectors:source-looker:airbyteDocker' uses this output of task ':airbyte-integrations:connectors:source-looker:test' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.2/userguide/validation_problems.html#implicit_dependency for more details about this problem.
Sending build context to Docker daemon  167.4kB

Step 1/18 : FROM python:3.7.11-alpine3.14 as base
3.7.11-alpine3.14: Pulling from library/python
a0d0a0d46f8b: Pulling fs layer
c11246b421be: Pulling fs layer
7f077e365186: Pulling fs layer
e0c8b5e7aa63: Pulling fs layer
08feaf68e62e: Pulling fs layer
e0c8b5e7aa63: Waiting
08feaf68e62e: Waiting
c11246b421be: Verifying Checksum
c11246b421be: Download complete
a0d0a0d46f8b: Verifying Checksum
a0d0a0d46f8b: Download complete
7f077e365186: Verifying Checksum
7f077e365186: Download complete
e0c8b5e7aa63: Download complete
a0d0a0d46f8b: Pull complete
08feaf68e62e: Verifying Checksum
08feaf68e62e: Download complete
c11246b421be: Pull complete
7f077e365186: Pull complete
e0c8b5e7aa63: Pull complete
08feaf68e62e: Pull complete
Digest: sha256:fcf44c5aaf990b16a37f5ad9f27d98c5da8b9bf280e48fe3383820882926ddee
Status: Downloaded newer image for python:3.7.11-alpine3.14
 ---> ab6374cb8b8e
Step 2/18 : FROM base as builder
 ---> ab6374cb8b8e
Step 3/18 : WORKDIR /airbyte/integration_code
 ---> Running in 145d36dda05f
Removing intermediate container 145d36dda05f
 ---> b5058f9d1929
Step 4/18 : RUN apk --no-cache upgrade     && pip install --upgrade pip     && apk --no-cache add tzdata build-base
 ---> Running in 02e0c68e6bc6
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz
(1/1) Upgrading alpine-keys (2.3-r1 -> 2.4-r0)
OK: 11 MiB in 35 packages
Requirement already satisfied: pip in /usr/local/lib/python3.7/site-packages (21.2.4)
Collecting pip
  Downloading pip-21.3.1-py3-none-any.whl (1.7 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 21.2.4
    Uninstalling pip-21.2.4:
      Successfully uninstalled pip-21.2.4
Successfully installed pip-21.3.1
�[91mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
�[0mfetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz
(1/21) Installing libgcc (10.3.1_git20210424-r2)
(2/21) Installing libstdc++ (10.3.1_git20210424-r2)
(3/21) Installing binutils (2.35.2-r2)
(4/21) Installing libmagic (5.40-r1)
(5/21) Installing file (5.40-r1)
(6/21) Installing libgomp (10.3.1_git20210424-r2)
(7/21) Installing libatomic (10.3.1_git20210424-r2)
(8/21) Installing libgphobos (10.3.1_git20210424-r2)
(9/21) Installing gmp (6.2.1-r0)
(10/21) Installing isl22 (0.22-r0)
(11/21) Installing mpfr4 (4.1.0-r0)
(12/21) Installing mpc1 (1.2.1-r0)
(13/21) Installing gcc (10.3.1_git20210424-r2)
(14/21) Installing musl-dev (1.2.2-r3)
(15/21) Installing libc-dev (0.7.2-r3)
(16/21) Installing g++ (10.3.1_git20210424-r2)
(17/21) Installing make (4.3-r0)
(18/21) Installing fortify-headers (1.1-r1)
(19/21) Installing patch (2.7.6-r7)
(20/21) Installing build-base (0.5-r2)
(21/21) Installing tzdata (2021e-r0)
Executing busybox-1.33.1-r3.trigger
OK: 205 MiB in 56 packages
Removing intermediate container 02e0c68e6bc6
 ---> beb792da48e6
Step 5/18 : COPY setup.py ./
 ---> ef28c5f59578
Step 6/18 : RUN pip install --prefix=/install .
 ---> Running in f3a837b13681
Processing /airbyte/integration_code
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting airbyte-cdk~=0.1
  Downloading airbyte_cdk-0.1.32-py3-none-any.whl (66 kB)
Collecting pydantic~=1.6
  Downloading pydantic-1.8.2-py3-none-any.whl (126 kB)
Collecting vcrpy
  Downloading vcrpy-4.1.1-py2.py3-none-any.whl (40 kB)
Collecting PyYAML~=5.4
  Downloading PyYAML-5.4.1.tar.gz (175 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting backoff
  Downloading backoff-1.11.1-py2.py3-none-any.whl (13 kB)
Collecting Deprecated~=1.2
  Downloading Deprecated-1.2.13-py2.py3-none-any.whl (9.6 kB)
Collecting pendulum
  Downloading pendulum-2.1.2.tar.gz (81 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting jsonref~=0.2
  Downloading jsonref-0.2-py3-none-any.whl (9.3 kB)
Collecting jsonschema~=3.2.0
  Downloading jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Collecting requests
  Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting wrapt<2,>=1.10
  Downloading wrapt-1.13.2.tar.gz (48 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting attrs>=17.4.0
  Downloading attrs-21.2.0-py2.py3-none-any.whl (53 kB)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from jsonschema~=3.2.0->airbyte-cdk~=0.1->source-looker==0.0.0) (57.4.0)
Collecting six>=1.11.0
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting importlib-metadata
  Using cached importlib_metadata-4.8.1-py3-none-any.whl (17 kB)
Collecting pyrsistent>=0.14.0
  Downloading pyrsistent-0.18.0.tar.gz (104 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting typing-extensions>=3.7.4.3
  Using cached typing_extensions-3.10.0.2-py3-none-any.whl (26 kB)
Collecting pytzdata>=2020.1
  Downloading pytzdata-2020.1-py2.py3-none-any.whl (489 kB)
Collecting python-dateutil<3.0,>=2.6
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting idna<4,>=2.5
  Downloading idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
  Downloading charset_normalizer-2.0.7-py3-none-any.whl (38 kB)
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.7-py2.py3-none-any.whl (138 kB)
Collecting yarl
  Downloading yarl-1.7.0-cp37-cp37m-musllinux_1_1_x86_64.whl (291 kB)
Collecting zipp>=0.5
  Using cached zipp-3.6.0-py3-none-any.whl (5.3 kB)
Collecting multidict>=4.0
  Downloading multidict-5.2.0-cp37-cp37m-musllinux_1_1_x86_64.whl (139 kB)
Building wheels for collected packages: source-looker, PyYAML, pendulum, pyrsistent, wrapt
  Building wheel for source-looker (setup.py): started
  Building wheel for source-looker (setup.py): finished with status 'done'
  Created wheel for source-looker: filename=source_looker-0.0.0-py3-none-any.whl size=1188 sha256=29db72aeaa5c5b1571f5573b260a849dd5183ea1ec6471e5cb7a0a38586fc9e2
  Stored in directory: /tmp/pip-ephem-wheel-cache-7pqalf7g/wheels/ae/94/dc/1c707c3e6652ba466c448a71a5a8bc364aeb95358e9f9979e6
  Building wheel for PyYAML (pyproject.toml): started
  Building wheel for PyYAML (pyproject.toml): finished with status 'done'
  Created wheel for PyYAML: filename=PyYAML-5.4.1-cp37-cp37m-linux_x86_64.whl size=45657 sha256=f4c4be2bd1f5048ceb72d144465091588945963c4023cca4e1737611265703fd
  Stored in directory: /root/.cache/pip/wheels/f4/51/cc/858604f7bb9cab887106ff266a541546ad783b4fa20875051d
  Building wheel for pendulum (pyproject.toml): started
  Building wheel for pendulum (pyproject.toml): finished with status 'done'
  Created wheel for pendulum: filename=pendulum-2.1.2-cp37-cp37m-linux_x86_64.whl size=126741 sha256=503cecf651f9b5db60f5f265ae5e752c1697cd85d0d8bbb162dcd4345c9aaab5
  Stored in directory: /root/.cache/pip/wheels/c4/67/26/134c5eca211ca5d86d51881b4f1e857268f1e793db490bd3d5
  Building wheel for pyrsistent (pyproject.toml): started
  Building wheel for pyrsistent (pyproject.toml): finished with status 'done'
  Created wheel for pyrsistent: filename=pyrsistent-0.18.0-cp37-cp37m-linux_x86_64.whl size=72916 sha256=e6ef85c685ed70613b5d01e5ddabb5b0a8ac8587ddefa547c16a92f5cb83af0c
  Stored in directory: /root/.cache/pip/wheels/54/35/6e/9827c774ec40ae001ae2a2ef063268add65762c4014951e97c
  Building wheel for wrapt (setup.py): started
  Building wheel for wrapt (setup.py): finished with status 'done'
  Created wheel for wrapt: filename=wrapt-1.13.2-cp37-cp37m-linux_x86_64.whl size=34586 sha256=5a18db727db0e44fda9dfba3c6ac107fa274f867ab3737f2c8e637a280f37955
  Stored in directory: /root/.cache/pip/wheels/31/1d/f6/e75f8e26fad04556fdff949623f3f7685c8933b29dac249645
Successfully built source-looker PyYAML pendulum pyrsistent wrapt
Installing collected packages: zipp, typing-extensions, six, multidict, idna, yarl, wrapt, urllib3, PyYAML, pytzdata, python-dateutil, pyrsistent, importlib-metadata, charset-normalizer, certifi, attrs, vcrpy, requests, pydantic, pendulum, jsonschema, jsonref, Deprecated, backoff, airbyte-cdk, source-looker
Successfully installed Deprecated-1.2.13 PyYAML-5.4.1 airbyte-cdk-0.1.32 attrs-21.2.0 backoff-1.11.1 certifi-2021.10.8 charset-normalizer-2.0.7 idna-3.3 importlib-metadata-4.8.1 jsonref-0.2 jsonschema-3.2.0 multidict-5.2.0 pendulum-2.1.2 pydantic-1.8.2 pyrsistent-0.18.0 python-dateutil-2.8.2 pytzdata-2020.1 requests-2.26.0 six-1.16.0 source-looker-0.0.0 typing-extensions-3.10.0.2 urllib3-1.26.7 vcrpy-4.1.1 wrapt-1.13.2 yarl-1.7.0 zipp-3.6.0
�[91mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
�[0mRemoving intermediate container f3a837b13681
 ---> ecb991706184
Step 7/18 : FROM base
 ---> ab6374cb8b8e
Step 8/18 : WORKDIR /airbyte/integration_code
 ---> Using cache
 ---> b5058f9d1929
Step 9/18 : COPY --from=builder /install /usr/local
 ---> 714b71734f9e
Step 10/18 : COPY --from=builder /usr/share/zoneinfo/Etc/UTC /etc/localtime
 ---> e5496015ba24
Step 11/18 : RUN echo "Etc/UTC" > /etc/timezone
 ---> Running in 0796e7b076dc
Removing intermediate container 0796e7b076dc
 ---> e26c1d4a10f1
Step 12/18 : RUN apk --no-cache add bash
 ---> Running in a8940fef1ebb
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz
(1/1) Installing bash (5.1.4-r0)
Executing bash-5.1.4-r0.post-install
Executing busybox-1.33.1-r3.trigger
OK: 13 MiB in 36 packages
Removing intermediate container a8940fef1ebb
 ---> d37b8d79772c
Step 13/18 : COPY main.py ./
 ---> 15deb4ad3ea4
Step 14/18 : COPY source_looker ./source_looker
 ---> 9d202b43c14e
Step 15/18 : ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
 ---> Running in 248f24f571dc
Removing intermediate container 248f24f571dc
 ---> a26ca74f7957
Step 16/18 : ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
 ---> Running in db273e34af6a
Removing intermediate container db273e34af6a
 ---> 500bf8341979
Step 17/18 : LABEL io.airbyte.version=0.2.5
 ---> Running in 1135cae04675
Removing intermediate container 1135cae04675
 ---> 865d77342370
Step 18/18 : LABEL io.airbyte.name=airbyte/source-looker
 ---> Running in 51b894c78d69
Removing intermediate container 51b894c78d69
 ---> 907835407a94
[Warning] One or more build-args [JDK_VERSION] were not consumed
Successfully built 907835407a94
Successfully tagged airbyte/source-looker:dev

> Task :airbyte-integrations:connectors:source-looker:test NO-SOURCE
> Task :airbyte-integrations:connectors:source-looker:jacocoTestReport SKIPPED
> Task :airbyte-integrations:connectors:source-looker:check UP-TO-DATE
> Task :airbyte-integrations:connectors:source-looker:build

> Task :airbyte-integrations:connectors:source-looker:sourceAcceptanceTest
============================= test session starts ==============================
platform linux -- Python 3.7.11, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /test_input
plugins: timeout-1.4.2, sugar-0.9.4
collected 12 items / 1 skipped / 11 selected

test_core.py ...........                                                 [ 91%]
test_full_refresh.py .                                                   [100%]

================== 12 passed, 1 skipped in 124.46s (0:02:04) ===================

> Task :airbyte-integrations:connectors:source-looker:integrationTest

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD SUCCESSFUL in 4m 11s
81 actionable tasks: 65 executed, 16 up-to-date

Publishing build scan...
https://gradle.com/s/vcagebll4sn2i

@yevhenii-ldv yevhenii-ldv temporarily deployed to more-secrets October 27, 2021 14:27 Inactive
@jrhizor jrhizor temporarily deployed to more-secrets October 27, 2021 14:28 Inactive
@yevhenii-ldv
Copy link
Contributor Author

yevhenii-ldv commented Oct 27, 2021

/publish connector=connectors/source-looker

🕑 connectors/source-looker https://github.com/airbytehq/airbyte/actions/runs/1390605825
✅ connectors/source-looker https://github.com/airbytehq/airbyte/actions/runs/1390605825

@jrhizor jrhizor temporarily deployed to more-secrets October 27, 2021 14:38 Inactive
@yevhenii-ldv yevhenii-ldv merged commit 62832cd into master Oct 27, 2021
@yevhenii-ldv yevhenii-ldv deleted the ykurochkin/source-looker-refactor-to-use-cdk branch October 27, 2021 14:46
schlattk pushed a commit to schlattk/airbyte that referenced this pull request Jan 4, 2022
…ing (airbytehq#7284)

* Migrate Looker source to CDK structure, add SAT testing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Source Looker: Refactor to use the CDK
4 participants