Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docker-compose up -d error #51

Open
quantum-booty opened this issue Aug 19, 2021 · 9 comments
Open

docker-compose up -d error #51

quantum-booty opened this issue Aug 19, 2021 · 9 comments

Comments

@quantum-booty
Copy link

quantum-booty commented Aug 19, 2021 •
edited

OS: Arcolinux

I have installed the prerequisites, not sure why I'm getting this error. I don't have Postgres installed, maybe it has to do with this?

➜ docker-compose up -d
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.9/http/client.py", line 1257, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1303, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1252, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1012, in _send_output
    self.send(msg)
  File "/usr/lib/python3.9/http/client.py", line 952, in send
    self.connect()
  File "/usr/lib/python3.9/site-packages/docker/transport/unixconn.py", line 43, in connect
    sock.connect(self.unix_socket)
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3.9/site-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/lib/python3.9/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.9/http/client.py", line 1257, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1303, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1252, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1012, in _send_output
    self.send(msg)
  File "/usr/lib/python3.9/http/client.py", line 952, in send
    self.connect()
  File "/usr/lib/python3.9/site-packages/docker/transport/unixconn.py", line 43, in connect
    sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/docker/api/client.py", line 214, in _retrieve_server_version
    return self.version(api_version=False)["ApiVersion"]
  File "/usr/lib/python3.9/site-packages/docker/api/daemon.py", line 181, in version
    return self._result(self._get(url), json=True)
  File "/usr/lib/python3.9/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/lib/python3.9/site-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/lib/python3.9/site-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.9/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/docker-compose", line 33, in <module>
    sys.exit(load_entry_point('docker-compose==1.29.2', 'console_scripts', 'docker-compose')())
  File "/usr/lib/python3.9/site-packages/compose/cli/main.py", line 81, in main
    command_func()
  File "/usr/lib/python3.9/site-packages/compose/cli/main.py", line 200, in perform_command
    project = project_from_options('.', options)
  File "/usr/lib/python3.9/site-packages/compose/cli/command.py", line 60, in project_from_options
    return get_project(
  File "/usr/lib/python3.9/site-packages/compose/cli/command.py", line 152, in get_project
    client = get_client(
  File "/usr/lib/python3.9/site-packages/compose/cli/docker_client.py", line 41, in get_client
    client = docker_client(
  File "/usr/lib/python3.9/site-packages/compose/cli/docker_client.py", line 170, in docker_client
    client = APIClient(use_ssh_client=not use_paramiko_ssh, **kwargs)
  File "/usr/lib/python3.9/site-packages/docker/api/client.py", line 197, in __init__
    self._version = self._retrieve_server_version()
  File "/usr/lib/python3.9/site-packages/docker/api/client.py", line 221, in _retrieve_server_version
    raise DockerException(
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
@DriLLFreAK100
Copy link

DriLLFreAK100 commented Aug 19, 2021 •
edited

I just cloned the repo a moment ago and ran into some errors when trying to run "docker-compose up" too. For my case, the error message was complaining about some missing mysql file towards the end..

I updated the docker image for puckel/docker-airflow to version 1.10.9 instead of 1.10.1 at the docker-compose.yml file. And also the sqlalchemy dependency to version 1.3.0.

It works after that on my machine. I'm on Windows 10 with Docker Desktop v3.5.2(66501). Hope this helps!

p.s. Also, I happened to miss out the 3rd prerequisite initially, that it to install the Apache Airflow python package. Don't miss it out.

My working version of the docker-compose.yml:

version: '3'
services:
  postgres:
    image: postgres:9.6
    environment:
      - POSTGRES_USER=airflow
      - POSTGRES_PASSWORD=airflow
      - POSTGRES_DB=airflow
    ports:
      - "5432:5432"

  webserver:
    image: puckel/docker-airflow:1.10.9
    build:
      context: https://github.com/puckel/docker-airflow.git#1.10.9
      dockerfile: Dockerfile
      args:
        AIRFLOW_DEPS: gcp_api,s3
        PYTHON_DEPS: sqlalchemy==1.3.0
    restart: always
    depends_on:
      - postgres
    environment:
      - LOAD_EX=n
      - EXECUTOR=Local
      - FERNET_KEY=jsDPRErfv8Z_eVTnGfF8ywd19j4pyqE3NpdUBA_oRTo=
    volumes:
      - ./examples/intro-example/dags:/usr/local/airflow/dags
      # Uncomment to include custom plugins
      # - ./plugins:/usr/local/airflow/plugins
    ports:
      - "8080:8080"
    command: webserver
    healthcheck:
      test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
      interval: 30s
      timeout: 30s
      retries: 3

@carlosvinter
Copy link

I applied your suggestion but I keep getting the same error with _mysql.
Docker engine v20.10.8
Docker desktop v 3.6.0

webserver_1 |
webserver_1 | Traceback (most recent call last):
webserver_1 | File "/usr/local/bin/airflow", line 21, in
webserver_1 | from airflow import configuration
webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/init.py", line 36, in
webserver_1 | from airflow import settings
webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 261, in
webserver_1 | configure_adapters()
webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 218, in configure_adapters
webserver_1 | import MySQLdb.converters
webserver_1 | File "/usr/local/lib/python3.6/site-packages/MySQLdb/init.py", line 24, in
webserver_1 | version_info, _mysql.version_info, _mysql.file
webserver_1 | NameError: name '_mysql' is not defined

@robertdefilippi
Copy link

I just cloned the repo a moment ago and ran into some errors when trying to run "docker-compose up" too. For my case, the error message was complaining about some missing mysql file towards the end..

I updated the docker image for puckel/docker-airflow to version 1.10.9 instead of 1.10.1 at the docker-compose.yml file. And also the sqlalchemy dependency to version 1.3.0.

It works after that on my machine. I'm on Windows 10 with Docker Desktop v3.5.2(66501). Hope this helps!

p.s. Also, I happened to miss out the 3rd prerequisite initially, that it to install the Apache Airflow python package. Don't miss it out.

My working version of the docker-compose.yml:

version: '3'
services:
  postgres:
    image: postgres:9.6
    environment:
      - POSTGRES_USER=airflow
      - POSTGRES_PASSWORD=airflow
      - POSTGRES_DB=airflow
    ports:
      - "5432:5432"

  webserver:
    image: puckel/docker-airflow:1.10.9
    build:
      context: https://github.com/puckel/docker-airflow.git#1.10.9
      dockerfile: Dockerfile
      args:
        AIRFLOW_DEPS: gcp_api,s3
        PYTHON_DEPS: sqlalchemy==1.3.0
    restart: always
    depends_on:
      - postgres
    environment:
      - LOAD_EX=n
      - EXECUTOR=Local
      - FERNET_KEY=jsDPRErfv8Z_eVTnGfF8ywd19j4pyqE3NpdUBA_oRTo=
    volumes:
      - ./examples/intro-example/dags:/usr/local/airflow/dags
      # Uncomment to include custom plugins
      # - ./plugins:/usr/local/airflow/plugins
    ports:
      - "8080:8080"
    command: webserver
    healthcheck:
      test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
      interval: 30s
      timeout: 30s
      retries: 3

This worked 😄

@zishen1229
Copy link

I applied your suggestion but I keep getting the same error with _mysql. Docker engine v20.10.8 Docker desktop v 3.6.0

webserver_1 | webserver_1 | Traceback (most recent call last): webserver_1 | File "/usr/local/bin/airflow", line 21, in webserver_1 | from airflow import configuration webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/init.py", line 36, in webserver_1 | from airflow import settings webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 261, in webserver_1 | configure_adapters() webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 218, in configure_adapters webserver_1 | import MySQLdb.converters webserver_1 | File "/usr/local/lib/python3.6/site-packages/MySQLdb/init.py", line 24, in webserver_1 | version_info, _mysql.version_info, _mysql.file webserver_1 | NameError: name '_mysql' is not defined

Having the same issue here. Did you find a way to fix it?

@DriLLFreAK100
Copy link

@zishen1229 Have you fulfilled the third prerequisite yet? That is to install the Python package for Apache Airflow here .

@zishen1229
Copy link

zishen1229 commented Oct 7, 2021 •
edited

@zishen1229 Have you fulfilled the third prerequisite yet? That is to install the Python package for Apache Airflow here .

Yes, my versions are as below:
docker:
Client:
Cloud integration: 1.0.17
Version: 20.10.8
API version: 1.41
Go version: go1.16.6
Git commit: 3967b7d
Built: Fri Jul 30 19:55:20 2021
OS/Arch: darwin/amd64
Context: default
Experimental: true
docker-compose:
docker-py version: 4.4.4
CPython version: 3.8.5
OpenSSL version: OpenSSL 1.1.1h 22 Sep 2020
airflow version: 2.1.4

@sophiaxc sophiaxc mentioned this issue Oct 18, 2021
Open
@tnanhd
Copy link

tnanhd commented Nov 4, 2021

@zishen1229
Hi. Have you fixed the error?
After a few tries, I ran it successfully by following the guide from @DriLLFreAK100 above.
Please notice that you have to modify the context to 1.10.9 as well.
context: https://github.com/puckel/docker-airflow.git#1.10.9

@ApplesHaveFeelings ApplesHaveFeelings mentioned this issue Jan 7, 2022
Open
@shreyanshumalviya
Copy link

For me Manual installation steps for older versions of WSL worked.

@KiranQureshi53 KiranQureshi53 mentioned this issue Jun 24, 2022
Open
@rns4731
Copy link

rns4731 commented Feb 22, 2024

This is what worked for me. I had to updated the PYTHON_DEPS

  webserver:
    image: puckel/docker-airflow:1.10.9
    build:
      context: https://github.com/puckel/docker-airflow.git#1.10.9
      dockerfile: Dockerfile
      args:
        AIRFLOW_DEPS: gcp_api,s3
        PYTHON_DEPS: sqlalchemy==1.3.0 MarkupSafe==2.0.1 wtforms<=2.3.3
    restart: always
    depends_on:
      - postgres
    environment:
      - LOAD_EX=n
      - EXECUTOR=Local
      - FERNET_KEY=jsDPRErfv8Z_eVTnGfF8ywd19j4pyqE3NpdUBA_oRTo=
    volumes:
      - ./examples/intro-example/dags:/usr/local/airflow/dags
      # Uncomment to include custom plugins
      # - ./plugins:/usr/local/airflow/plugins
    ports:
      - "8080:8080"
    command: webserver
    healthcheck:
      test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
      interval: 30s
      timeout: 30s
      retries: 3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@rns4731 @robertdefilippi @DriLLFreAK100 @quantum-booty @zishen1229 @shreyanshumalviya @tnanhd @carlosvinter