Experiment B #2

b-long · 2022-09-22T00:02:32Z

Build Results

Succeeds: No
Fast? No

Flows working?

basic-add-flow: N/A
basic-pandas-flow: N/A

Notable output from build (see full log below):

ModuleNotFoundError: No module named 'aid'
The virtual environment found in /workdir/.venv seems to be broken.
Prefect found by pip show?:

Step 14/19 : RUN pip show prefect || pip install git+https://github.com/PrefectHQ/prefect.git@1.3.1#egg=prefect[all_orchestration_extras]
  ---> Running in 4ed906b8ca5e
  Name: prefect
  Version: 1.3.1Step 14/19 : RUN pip show prefect || pip install 
 git+https://github.com/PrefectHQ/prefect.git@1.3.1#egg=prefect[all_orchestration_extras]
   ---> Running in 4ed906b8ca5e
   Name: prefect
   Version: 1.3.1

Full build log:

$ ./aid/prefect/step-3-register-flow.sh
Configuration file exists at /Users/b-long/Library/Application Support/pypoetry, reusing this directory.

Consider moving configuration to /Users/b-long/Library/Preferences/pypoetry, as support for the legacy directory will be removed in an upcoming release.
/Users/b-long/Desktop/github/b-long/aid/.venv/lib/python3.10/site-packages/prefect/storage/docker.py:325: UserWarning: This Docker storage object has no `registry_url`, and will not be pushed.
  self._build_image(push=push)
[2022-09-20 21:49:30-0400] INFO - prefect.Docker | Building the flow's Docker storage...
Step 1/19 : FROM prefecthq/prefect:1.3.1-python3.10 as base
 ---> 16d2aaf55962
Step 2/19 : RUN apt-get update && apt-get install -y gcc python3-dev git gcc g++ libblas-dev
 ---> Using cache
 ---> b9f70fa0bb64
Step 3/19 : RUN pip install --upgrade pip
 ---> Using cache
 ---> a5996d1538f4
Step 4/19 : RUN pip install poetry
 ---> Using cache
 ---> 820c6d3ed47f
Step 5/19 : FROM base
 ---> 820c6d3ed47f
Step 6/19 : RUN poetry config virtualenvs.create false
 ---> Running in 5e68a5e7b5b6
Removing intermediate container 5e68a5e7b5b6
 ---> 72c9f3c439d2
Step 7/19 : RUN poetry config virtualenvs.prefer-active-python true
 ---> Running in 694c175ac4bf
Removing intermediate container 694c175ac4bf
 ---> 670d098ac9e2
Step 8/19 : RUN poetry config installer.no-binary :all:
 ---> Running in 1ce937e04244
Removing intermediate container 1ce937e04244
 ---> 5dc88b746071
Step 9/19 : WORKDIR /workdir/
 ---> Running in 9eac606a4c1a
Removing intermediate container 9eac606a4c1a
 ---> fbcea550f99a
Step 10/19 : COPY . /workdir/
 ---> 6eda74efb3be
Step 11/19 : RUN poetry install --only main --no-root
 ---> Running in 47fd766539ca
The virtual environment found in /workdir/.venv seems to be broken.

Recreating virtualenv aid in /workdir/.venv
Installing dependencies from lock file
Package operations: 66 installs, 0 updates, 0 removals
  • Installing certifi (2022.6.15.1)
  • Installing charset-normalizer (2.1.1)
  • Installing frozenlist (1.3.1)
  • Installing idna (3.3)
  • Installing locket (1.0.0)
  • Installing multidict (6.0.2)
  • Installing pyasn1 (0.4.8)
  • Installing pyparsing (3.0.9)
  • Installing toolz (0.12.0)
  • Installing urllib3 (1.26.12)
  • Installing aiosignal (1.2.0)
  • Installing async-timeout (4.0.2)
  • Installing attrs (22.1.0)
  • Installing cachetools (5.2.0)
  • Installing fsspec (2022.8.2)
  • Installing cloudpickle (2.2.0)
  • Installing oauthlib (3.2.1)
  • Installing heapdict (1.0.1)
  • Installing packaging (21.3)
  • Installing markupsafe (2.1.1)
  • Installing partd (1.3.0)
  • Installing pyasn1-modules (0.2.8)
  • Installing pyyaml (6.0)
  • Installing requests (2.28.1)
  • Installing rsa (4.9)
  • Installing six (1.16.0)
  • Installing yarl (1.8.1)
  • Installing aiohttp (3.8.1)
  • Installing click (8.1.3)
  • Installing dask (2022.9.0)
  • Installing google-auth (2.11.0)
  • Installing jinja2 (3.1.2)
  • Installing msgpack (1.0.4)
  • Installing python-dateutil (2.8.2)
  • Installing python-json-logger (2.0.4)
  • Installing psutil (5.9.2)
  • Installing iso8601 (1.0.2)
  • Installing requests-oauthlib (1.3.1)
  • Installing sortedcontainers (2.4.0)
  • Installing tblib (1.7.0)
  • Installing tornado (6.1)
  • Installing typing-extensions (4.3.0)
  • Installing websocket-client (1.4.1)
  • Installing zict (2.2.0)
  • Installing distributed (2022.9.0)
  • Installing kopf (1.35.6)
  • Installing kubernetes (24.2.0)
  • Installing marshmallow (3.17.1)
  • Installing pytzdata (2020.1)
  • Installing text-unidecode (1.3)
  • Installing kubernetes-asyncio (24.2.2)
  • Installing croniter (1.3.7)
  • Installing dask-kubernetes (2022.7.0)
  • Installing docker (6.0.0)
  • Installing importlib-resources (5.9.0)
  • Installing marshmallow-oneofschema (3.0.1)
  • Installing mypy-extensions (0.4.3)
  • Installing numpy (1.23.3)
  • Installing pendulum (2.1.2)
  • Installing python-box (6.0.2)
  • Installing python-slugify (6.1.2)
  • Installing pytz (2022.2.1)
  • Installing tabulate (0.8.10)
  • Installing toml (0.10.2)
  • Installing pandas (1.4.4)
  • Installing prefect (1.3.1)

Removing intermediate container 47fd766539ca
 ---> bdecee499dac
Step 12/19 : ENV PREFECT__USER_CONFIG_PATH='/opt/prefect/config.toml'
 ---> Running in 82dca23fdc64
Removing intermediate container 82dca23fdc64
 ---> af11d0c520a3
Step 13/19 : RUN pip install pip --upgrade
 ---> Running in 896738a67f1c
Requirement already satisfied: pip in /usr/local/lib/python3.10/site-packages (22.2.2)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Removing intermediate container 896738a67f1c
 ---> 87acb09d6830
Step 14/19 : RUN pip show prefect || pip install git+https://github.com/PrefectHQ/prefect.git@1.3.1#egg=prefect[all_orchestration_extras]
 ---> Running in 4ed906b8ca5e
Name: prefect
Version: 1.3.1
Summary: The Prefect Core automation and scheduling engine.
Home-page: https://www.github.com/PrefectHQ/prefect
Author: Prefect Technologies, Inc.
Author-email: help@prefect.io
License: Apache License 2.0
Location: /usr/local/lib/python3.10/site-packages
Requires: click, cloudpickle, croniter, dask, distributed, docker, importlib-resources, marshmallow, marshmallow-oneofschema, msgpack, mypy-extensions, packaging, pendulum, python-box, python-dateutil, python-slugify, pytz, pyyaml, requests, tabulate, toml, urllib3
Required-by: 
Removing intermediate container 4ed906b8ca5e
 ---> c4df40cebad4
Step 15/19 : RUN pip install 'wheel'
 ---> Running in da0acd612c45
Requirement already satisfied: wheel in /usr/local/lib/python3.10/site-packages (0.37.1)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Removing intermediate container da0acd612c45
 ---> f1d5396f67ac
Step 16/19 : RUN mkdir -p /opt/prefect/
 ---> Running in b3688e1c0da6
Removing intermediate container b3688e1c0da6
 ---> f6eb158f903f
Step 17/19 : COPY ./tmpds0kp8lw/basic-pandas-flow.flow /opt/prefect/flows/basic-pandas-flow.prefect
 ---> d44c4a56c57f
Step 18/19 : COPY ./tmpds0kp8lw/healthcheck.py /opt/prefect/healthcheck.py
 ---> 4cb6b069d7ea
Step 19/19 : RUN python /opt/prefect/healthcheck.py '["/opt/prefect/flows/basic-pandas-flow.prefect"]' '(3, 10)'
 ---> Running in 238cfacbeb80
Beginning health checks...
System Version check: OK
/opt/prefect/healthcheck.py:130: UserWarning: Flow uses module which is not importable. Refer to documentation on how to import custom modules https://docs.prefect.io/api/latest/storage.html#docker
  flows = cloudpickle_deserialization_check(flow_file_paths)
Traceback (most recent call last):
  File "/opt/prefect/healthcheck.py", line 130, in <module>
    flows = cloudpickle_deserialization_check(flow_file_paths)
  File "/opt/prefect/healthcheck.py", line 43, in cloudpickle_deserialization_check
    flows.append(cloudpickle.loads(flow_bytes))
ModuleNotFoundError: No module named 'aid'

The command '/bin/sh -c python /opt/prefect/healthcheck.py '["/opt/prefect/flows/basic-pandas-flow.prefect"]' '(3, 10)'' returned a non-zero code: 1
Flow URL: http://localhost:8080/aid-tenant/flow/1dd5584d-36da-4a09-8d91-dd498c1af316
 └── ID: e7f240b4-4ff7-4c88-864d-0bef9c9c97c7
 └── Project: aid-tenant
 └── Labels: []$ ./aid/prefect/step-3-register-flow.sh
Configuration file exists at /Users/b-long/Library/Application Support/pypoetry, reusing this directory.

Consider moving configuration to /Users/b-long/Library/Preferences/pypoetry, as support for the legacy directory will be removed in an upcoming release.
/Users/b-long/Desktop/github/b-long/aid/.venv/lib/python3.10/site-packages/prefect/storage/docker.py:325: UserWarning: This Docker storage object has no `registry_url`, and will not be pushed.
  self._build_image(push=push)
[2022-09-20 21:49:30-0400] INFO - prefect.Docker | Building the flow's Docker storage...
Step 1/19 : FROM prefecthq/prefect:1.3.1-python3.10 as base
 ---> 16d2aaf55962
Step 2/19 : RUN apt-get update && apt-get install -y gcc python3-dev git gcc g++ libblas-dev
 ---> Using cache
 ---> b9f70fa0bb64
Step 3/19 : RUN pip install --upgrade pip
 ---> Using cache
 ---> a5996d1538f4
Step 4/19 : RUN pip install poetry
 ---> Using cache
 ---> 820c6d3ed47f
Step 5/19 : FROM base
 ---> 820c6d3ed47f
Step 6/19 : RUN poetry config virtualenvs.create false
 ---> Running in 5e68a5e7b5b6
Removing intermediate container 5e68a5e7b5b6
 ---> 72c9f3c439d2
Step 7/19 : RUN poetry config virtualenvs.prefer-active-python true
 ---> Running in 694c175ac4bf
Removing intermediate container 694c175ac4bf
 ---> 670d098ac9e2
Step 8/19 : RUN poetry config installer.no-binary :all:
 ---> Running in 1ce937e04244
Removing intermediate container 1ce937e04244
 ---> 5dc88b746071
Step 9/19 : WORKDIR /workdir/
 ---> Running in 9eac606a4c1a
Removing intermediate container 9eac606a4c1a
 ---> fbcea550f99a
Step 10/19 : COPY . /workdir/
 ---> 6eda74efb3be
Step 11/19 : RUN poetry install --only main --no-root
 ---> Running in 47fd766539ca
The virtual environment found in /workdir/.venv seems to be broken.

Recreating virtualenv aid in /workdir/.venv
Installing dependencies from lock file
Package operations: 66 installs, 0 updates, 0 removals
  • Installing certifi (2022.6.15.1)
  • Installing charset-normalizer (2.1.1)
  • Installing frozenlist (1.3.1)
  • Installing idna (3.3)
  • Installing locket (1.0.0)
  • Installing multidict (6.0.2)
  • Installing pyasn1 (0.4.8)
  • Installing pyparsing (3.0.9)
  • Installing toolz (0.12.0)
  • Installing urllib3 (1.26.12)
  • Installing aiosignal (1.2.0)
  • Installing async-timeout (4.0.2)
  • Installing attrs (22.1.0)
  • Installing cachetools (5.2.0)
  • Installing fsspec (2022.8.2)
  • Installing cloudpickle (2.2.0)
  • Installing oauthlib (3.2.1)
  • Installing heapdict (1.0.1)
  • Installing packaging (21.3)
  • Installing markupsafe (2.1.1)
  • Installing partd (1.3.0)
  • Installing pyasn1-modules (0.2.8)
  • Installing pyyaml (6.0)
  • Installing requests (2.28.1)
  • Installing rsa (4.9)
  • Installing six (1.16.0)
  • Installing yarl (1.8.1)
  • Installing aiohttp (3.8.1)
  • Installing click (8.1.3)
  • Installing dask (2022.9.0)
  • Installing google-auth (2.11.0)
  • Installing jinja2 (3.1.2)
  • Installing msgpack (1.0.4)
  • Installing python-dateutil (2.8.2)
  • Installing python-json-logger (2.0.4)
  • Installing psutil (5.9.2)
  • Installing iso8601 (1.0.2)
  • Installing requests-oauthlib (1.3.1)
  • Installing sortedcontainers (2.4.0)
  • Installing tblib (1.7.0)
  • Installing tornado (6.1)
  • Installing typing-extensions (4.3.0)
  • Installing websocket-client (1.4.1)
  • Installing zict (2.2.0)
  • Installing distributed (2022.9.0)
  • Installing kopf (1.35.6)
  • Installing kubernetes (24.2.0)
  • Installing marshmallow (3.17.1)
  • Installing pytzdata (2020.1)
  • Installing text-unidecode (1.3)
  • Installing kubernetes-asyncio (24.2.2)
  • Installing croniter (1.3.7)
  • Installing dask-kubernetes (2022.7.0)
  • Installing docker (6.0.0)
  • Installing importlib-resources (5.9.0)
  • Installing marshmallow-oneofschema (3.0.1)
  • Installing mypy-extensions (0.4.3)
  • Installing numpy (1.23.3)
  • Installing pendulum (2.1.2)
  • Installing python-box (6.0.2)
  • Installing python-slugify (6.1.2)
  • Installing pytz (2022.2.1)
  • Installing tabulate (0.8.10)
  • Installing toml (0.10.2)
  • Installing pandas (1.4.4)
  • Installing prefect (1.3.1)

Removing intermediate container 47fd766539ca
 ---> bdecee499dac
Step 12/19 : ENV PREFECT__USER_CONFIG_PATH='/opt/prefect/config.toml'
 ---> Running in 82dca23fdc64
Removing intermediate container 82dca23fdc64
 ---> af11d0c520a3
Step 13/19 : RUN pip install pip --upgrade
 ---> Running in 896738a67f1c
Requirement already satisfied: pip in /usr/local/lib/python3.10/site-packages (22.2.2)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Removing intermediate container 896738a67f1c
 ---> 87acb09d6830
Step 14/19 : RUN pip show prefect || pip install git+https://github.com/PrefectHQ/prefect.git@1.3.1#egg=prefect[all_orchestration_extras]
 ---> Running in 4ed906b8ca5e
Name: prefect
Version: 1.3.1
Summary: The Prefect Core automation and scheduling engine.
Home-page: https://www.github.com/PrefectHQ/prefect
Author: Prefect Technologies, Inc.
Author-email: help@prefect.io
License: Apache License 2.0
Location: /usr/local/lib/python3.10/site-packages
Requires: click, cloudpickle, croniter, dask, distributed, docker, importlib-resources, marshmallow, marshmallow-oneofschema, msgpack, mypy-extensions, packaging, pendulum, python-box, python-dateutil, python-slugify, pytz, pyyaml, requests, tabulate, toml, urllib3
Required-by: 
Removing intermediate container 4ed906b8ca5e
 ---> c4df40cebad4
Step 15/19 : RUN pip install 'wheel'
 ---> Running in da0acd612c45
Requirement already satisfied: wheel in /usr/local/lib/python3.10/site-packages (0.37.1)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Removing intermediate container da0acd612c45
 ---> f1d5396f67ac
Step 16/19 : RUN mkdir -p /opt/prefect/
 ---> Running in b3688e1c0da6
Removing intermediate container b3688e1c0da6
 ---> f6eb158f903f
Step 17/19 : COPY ./tmpds0kp8lw/basic-pandas-flow.flow /opt/prefect/flows/basic-pandas-flow.prefect
 ---> d44c4a56c57f
Step 18/19 : COPY ./tmpds0kp8lw/healthcheck.py /opt/prefect/healthcheck.py
 ---> 4cb6b069d7ea
Step 19/19 : RUN python /opt/prefect/healthcheck.py '["/opt/prefect/flows/basic-pandas-flow.prefect"]' '(3, 10)'
 ---> Running in 238cfacbeb80
Beginning health checks...
System Version check: OK
/opt/prefect/healthcheck.py:130: UserWarning: Flow uses module which is not importable. Refer to documentation on how to import custom modules https://docs.prefect.io/api/latest/storage.html#docker
  flows = cloudpickle_deserialization_check(flow_file_paths)
Traceback (most recent call last):
  File "/opt/prefect/healthcheck.py", line 130, in <module>
    flows = cloudpickle_deserialization_check(flow_file_paths)
  File "/opt/prefect/healthcheck.py", line 43, in cloudpickle_deserialization_check
    flows.append(cloudpickle.loads(flow_bytes))
ModuleNotFoundError: No module named 'aid'

The command '/bin/sh -c python /opt/prefect/healthcheck.py '["/opt/prefect/flows/basic-pandas-flow.prefect"]' '(3, 10)'' returned a non-zero code: 1
Flow URL: http://localhost:8080/aid-tenant/flow/1dd5584d-36da-4a09-8d91-dd498c1af316
 └── ID: e7f240b4-4ff7-4c88-864d-0bef9c9c97c7
 └── Project: aid-tenant
 └── Labels: []

b-long · 2022-10-29T13:03:41Z

Superseded by #5 .

b-long added 2 commits September 20, 2022 21:38

Add numpy

1ad762d

Experiment B

24d0565

b-long closed this Oct 29, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Experiment B #2

Experiment B #2

b-long commented Sep 22, 2022 •

edited

Loading

b-long commented Oct 29, 2022

Experiment B #2

Experiment B #2

Conversation

b-long commented Sep 22, 2022 • edited Loading

b-long commented Oct 29, 2022

b-long commented Sep 22, 2022 •

edited

Loading