Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker inspect fails when running with Docker in Docker #1978

Closed
robin-moss opened this issue Jul 15, 2021 · 10 comments · Fixed by #1997
Closed

Docker inspect fails when running with Docker in Docker #1978

robin-moss opened this issue Jul 15, 2021 · 10 comments · Fixed by #1997

Comments

@robin-moss
Copy link

@robin-moss robin-moss commented Jul 15, 2021

When trying to run pre commits with docker_image language via GitLabs Docker in Docker approach the docker inspect fails, and so pre-commit throws an error when trying to get the mount path.

The code for this already has a fall back of "return path" but doesn't handle the docker command failing.

Could the code for this be modified to handle the docker inspect failing?

More details of docker in docker are here

Full logs of the failure:

version information

pre-commit version: 2.13.0
sys.version:
    3.6.9 (default, Jan 26 2021, 15:33:00) 
    [GCC 8.4.0]
sys.executable: /usr/bin/python3
os.name: posix
sys.platform: linux

error information

An unexpected error has occurred: CalledProcessError: command: ('/usr/bin/docker', 'inspect', 'runner-dmrvfsu-project-35395-concurrent-0')
return code: 1
expected return code: 0
stdout:
    []
    
stderr:
    Error: No such object: runner-dmrvfsu-project-35395-concurrent-0
    
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/pre_commit/error_handler.py", line 65, in error_handler
    yield
  File "/usr/local/lib/python3.6/dist-packages/pre_commit/main.py", line 385, in main
    return run(args.config, store, args)
  File "/usr/local/lib/python3.6/dist-packages/pre_commit/commands/run.py", line 410, in run
    return _run_hooks(config, hooks, skips, args, environ)
  File "/usr/local/lib/python3.6/dist-packages/pre_commit/commands/run.py", line 288, in _run_hooks
    verbose=args.verbose, use_color=args.color,
  File "/usr/local/lib/python3.6/dist-packages/pre_commit/commands/run.py", line 194, in _run_single_hook
    retcode, out = language.run_hook(hook, filenames, use_color)
  File "/usr/local/lib/python3.6/dist-packages/pre_commit/languages/docker_image.py", line 19, in run_hook
    cmd = docker_cmd() + hook.cmd
  File "/usr/local/lib/python3.6/dist-packages/pre_commit/languages/docker.py", line 107, in docker_cmd
    '-v', f'{_get_docker_path(os.getcwd())}:/src:rw,Z',
  File "/usr/local/lib/python3.6/dist-packages/pre_commit/languages/docker.py", line 34, in _get_docker_path
    _, out, _ = cmd_output_b('docker', 'inspect', hostname)
  File "/usr/local/lib/python3.6/dist-packages/pre_commit/util.py", line 154, in cmd_output_b
    raise CalledProcessError(returncode, cmd, retcode, stdout_b, stderr_b)
pre_commit.util.CalledProcessError: command: ('/usr/bin/docker', 'inspect', 'runner-dmrvfsu-project-35395-concurrent-0')
return code: 1
expected return code: 0
stdout:
    []
    
stderr:
    Error: No such object: runner-dmrvfsu-project-35395-concurrent-0
@asottile
Copy link
Member

@asottile asottile commented Jul 15, 2021

I believe this is already fixed by #1919

@asottile asottile closed this Jul 15, 2021
@robin-moss
Copy link
Author

@robin-moss robin-moss commented Jul 15, 2021

Thanks, so just need to wait for the next release :)

@robin-moss
Copy link
Author

@robin-moss robin-moss commented Jul 15, 2021

Looking closer at those changes the docker inspect command will still fail for my case

@asottile
Copy link
Member

@asottile asottile commented Jul 15, 2021

@wiget
Copy link

@wiget wiget commented Jul 16, 2021

@asottile docker inspect is failing because job container is run on host Docker engine (and docker socket is not mounted), so docker client is querying Docker in Docker engine, where it can't find job container.

Job container is the place where pre-commit is run.

@asottile asottile reopened this Jul 16, 2021
@asottile
Copy link
Member

@asottile asottile commented Jul 16, 2021

can you provide commands I can run to validate?

@wiget
Copy link

@wiget wiget commented Jul 16, 2021

@asottile I prepared test repository: https://github.com/wiget/pre-commit-in-gitlab-with-dind
It uses docker-compose to simulate how GitLab CI with Docker in Docker works.
To test this you will need docker and docker-compose installed and then run in directory when above repository is cloned:

docker-compose up -d docker
docker-compose up --build job

You can tweak Dockerfile to install pre-commit from git for easier testing.

@asottile
Copy link
Member

@asottile asottile commented Jul 16, 2021

can you reproduce without docker compose? I'd rather not use it if I don't need to

@wiget
Copy link

@wiget wiget commented Jul 16, 2021

Sure. I added few scripts.
prepare.sh will create docker network, volumes, start Docker in Docker and build job container image.
test.sh will run job container showing what is wrong.
cleanup.sh should remove all created resources.

@robin-moss
Copy link
Author

@robin-moss robin-moss commented Aug 5, 2021

Thanks :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

3 participants