arion package build failure on Mac Os X ("OSError: AF_UNIX path too long" error when building docker)

[jbmusso]

Describe the bug

arion package doesn't build on Mac OS X using nixpkgs current master (2019-10-25). It could be a build failure in the python docker package in docker-compose's closure, as suggested by @roberth

To Reproduce

Minimal shell.nix file:

{ pkgs ? import <nixpkgs> {} }:
let 
  arionPkg = pkgs.callPackage (pkgs.fetchFromGitHub {
    owner = "NixOS";
    repo = "nixpkgs";
    rev = "ce14f092eab6efcb2915c1db189558c1d5a469e8"; # ~2019-10-25 Friday morning
    sha256 = "1w901v41gfnxd9xaanxns5qxrjnhlpfgsn6g7bhf76jg11lggxf8";
  }) {};
in arionPkg.arion

nix-shell eventually fails with a build error.

Expected behavior

arion and its dependencies (docker and docker-compose) should build on Mac OS.

Screenshots

n/a

Additional context

It appears that the path length limit for a Unix domain socket is around 104 on Mac OS. This could be related.

I'm also referencing #71092 where this issue was first discussed.

Full error log:

Installing collected packages: docker
Successfully installed docker-4.1.0
/private/var/folders/8c/cjp8r0355mbdsygc0_mg00l80000gn/T/nix-build-python3.7-docker-4.1.0.drv-0/docker-4.1.0
Finished executing pipInstallPhase
post-installation fixup
strip is /nix/store/ca22h96ag2fq0laa8v91vnv0pyy87sfc-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/qhxq3xvayyyjrh7sd4acxayg3i5i3g9i-python3.7-docker-4.1.0/lib
patching script interpreter paths in /nix/store/qhxq3xvayyyjrh7sd4acxayg3i5i3g9i-python3.7-docker-4.1.0
running install tests
============================= test session starts ==============================
platform darwin -- Python 3.7.5, pytest-5.2.1, py-1.8.0, pluggy-0.13.0
rootdir: /private/var/folders/8c/cjp8r0355mbdsygc0_mg00l80000gn/T/nix-build-python3.7-docker-4.1.0.drv-0/docker-4.1.0
collected 547 items

tests/unit/api_build_test.py .........s...                               [  2%]
tests/unit/api_container_test.py ....................................... [  9%]
.........................................................                [ 19%]
tests/unit/api_exec_test.py .....                                        [ 20%]
tests/unit/api_image_test.py .........................                   [ 25%]
tests/unit/api_network_test.py ......                                    [ 26%]
tests/unit/api_test.py .......................F.............             [ 33%]
tests/unit/api_volume_test.py ..........                                 [ 35%]
tests/unit/auth_test.py ................................................ [ 43%]
........                                                                 [ 45%]
tests/unit/client_test.py ..........                                     [ 47%]
tests/unit/dockertypes_test.py ......................................... [ 54%]
.....x............                                                       [ 57%]
tests/unit/errors_test.py ..................                             [ 61%]
tests/unit/models_containers_test.py ................................... [ 67%]
...                                                                      [ 68%]
tests/unit/models_images_test.py ................                        [ 71%]
tests/unit/models_networks_test.py ......                                [ 72%]
tests/unit/models_resources_test.py ..                                   [ 72%]
tests/unit/models_services_test.py .                                     [ 72%]
tests/unit/ssladapter_test.py ......                                     [ 73%]
tests/unit/swarm_test.py ...                                             [ 74%]
tests/unit/types_containers_test.py .                                    [ 74%]
tests/unit/utils_build_test.py ...s.......................s.........F... [ 82%]
...                                                                      [ 82%]
tests/unit/utils_config_test.py .....s....                               [ 84%]
tests/unit/utils_json_stream_test.py .......                             [ 85%]
tests/unit/utils_proxy_test.py .....                                     [ 86%]
tests/unit/utils_test.py ...........................................s... [ 95%]
..........................                                               [100%]

=================================== FAILURES ===================================
_______________ UnixSocketStreamTest.test_early_stream_response ________________

self = <tests.unit.api_test.UnixSocketStreamTest testMethod=test_early_stream_response>

    def setUp(self):
        socket_dir = tempfile.mkdtemp()
        self.build_context = tempfile.mkdtemp()
        self.addCleanup(shutil.rmtree, socket_dir)
        self.addCleanup(shutil.rmtree, self.build_context)
        self.socket_file = os.path.join(socket_dir, 'test_sock.sock')
>       self.server_socket = self._setup_socket()

tests/unit/api_test.py:372:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <tests.unit.api_test.UnixSocketStreamTest testMethod=test_early_stream_response>

    def _setup_socket(self):
        server_sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
>       server_sock.bind(self.socket_file)
E       OSError: AF_UNIX path too long

tests/unit/api_test.py:387: OSError
_________________________ TarTest.test_tar_socket_file _________________________

self = <tests.unit.utils_build_test.TarTest testMethod=test_tar_socket_file>

    @pytest.mark.skipif(IS_WINDOWS_PLATFORM, reason='No UNIX sockets on Win32')
    def test_tar_socket_file(self):
        base = tempfile.mkdtemp()
        self.addCleanup(shutil.rmtree, base)
        for d in ['foo', 'bar']:
            os.makedirs(os.path.join(base, d))
        sock = socket.socket(socket.AF_UNIX)
        self.addCleanup(sock.close)
>       sock.bind(os.path.join(base, 'test.sock'))
E       OSError: AF_UNIX path too long

tests/unit/utils_build_test.py:463: OSError
============= 2 failed, 539 passed, 5 skipped, 1 xfailed in 4.70s ==============
builder for '/nix/store/0j7mc6zhfq7x70n3cbl4wzplbn424y71-python3.7-docker-4.1.0.drv' failed with exit code 1
cannot build derivation '/nix/store/cqamkpf3gw05bk7sw163ysm53yqyfh2z-docker-compose-1.24.1.drv': 1 dependencies couldn't be built
error: build of '/nix/store/cqamkpf3gw05bk7sw163ysm53yqyfh2z-docker-compose-1.24.1.drv' failed

Metadata

nix-shell -p nix-info --run "nix-info -m"

 - system: `"x86_64-darwin"`
 - host os: `Darwin 18.6.0, macOS 10.14.5`
 - multi-user?: `no`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.3.1`
 - channels(jb): `"nixpkgs-20.03pre196811.7818f30cc4b, nixos-19.03-19.03.173408.bd6ba87381e"`
 - nixpkgs: `/Users/jb/.nix-defexpr/channels/nixpkgs`

nix run nixpkgs.nix-info -c nix-info -m failed with:

Assertion failed: (size_ < capacity_), function push_back, file src/libexpr/attr-set.hh, line 54.
[1]    54374 abort      nix run nixpkgs.nix-info -c nix-info -m

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute: arion
# a list of nixos modules affected by the problem
module: 

CC @jonringer - thanks for your time and effort. Happy to help in any way if I can.

[jonringer]

hmm, this is a darwin related issue @NixOS/darwin-maintainers

i think the fix might be _allowLocalNetworking = true; or something to that effect. I'll have to check after work

Thanks for submitting a ticket though :)

[veprbl]

This looks like an instance of NixOS/nix#1085 . We probably should just disable the failing tests.

[domenkozar]

now decorator package fails

[jbmusso]

Thanks for looking into it.
Current master builds fine, though on a different setup (MacBook's keyboard being, err, replaced):

nix run nixpkgs.nix-info -c nix-info -m

- system: `"x86_64-darwin"`
- host os: `Darwin 16.7.0, macOS 10.12.6`
- multi-user?: `yes`
- sandbox: `no`
- version: `nix-env (Nix) 2.3.1`
- channels(jb): `"nixpkgs-20.03pre197736.91d5b3f07d2"`
- channels(root): `"nixpkgs-18.09pre144572.9720833743f"`
- nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`

Using:

{ pkgs ? import <nixpkgs> {} }:
let 
  arionPkg = pkgs.callPackage (pkgs.fetchFromGitHub {
    owner = "NixOS";
    repo = "nixpkgs";
    rev = "3e639196de915247f71f3395c53ee2f52d472d34"; # 20191028 Monday
    sha256 = "1iyzqxz988xg78q8hh6sldy09n5h9hb6h7rsvl4vsncs792lzp5q";
  }) {};
in arionPkg.arion

[jbmusso]

I've gotten a bit farther, playing around with https://github.com/hercules-ci/arion/tree/8f2b953701420dc55f67e84d0023f686b6268e3b/examples/minimal (20191028 master)

# examples/minimal/arion-compose.nix

{ pkgs, ... }:
{
  config.docker-compose.services = {

    webserver = {
      service.useHostStore = true;
      service.command = [ "sh" "-c" ''
                  cd "$$WEB_ROOT"
                  ${pkgs.python3}/bin/python -m http.server
                '' ];
      service.ports = [
        "8000:8000" # host:container
      ];
      service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
    };
  };
}

And then arion up command fails with this error:

[nix-shell:~/nix]$ arion up
/nix/store/7sidzyl6x6ddd1wbq4dx31rnaczpmfi7-docker-compose.yaml
Starting ed32e7c9f0a4_nix_webserver_1 ... done
Attaching to ed32e7c9f0a4_nix_webserver_1
ed32e7c9f0a4_nix_webserver_1 | standard_init_linux.go:211: exec user process caused "exec format error"
ed32e7c9f0a4_nix_webserver_1 exited with code 1

Here's the content of /nix/store/7sidzyl6x6ddd1wbq4dx31rnaczpmfi7-docker-compose.yaml file being generated:

{
  "services": {
    "webserver": {
      "command": [
        "sh",
        "-c",
        "cd \"$$WEB_ROOT\"\n/nix/store/lqg58f3cnjrgwdhr70hwcf6b555iqnsf-python3-3.7.5/bin/python -m http.server\n"
      ],
      "environment": {
        "NIX_REMOTE": "",
        "WEB_ROOT": "/nix/store/xjdn9x3gncq27828hhck7hjk89id8mw8-nix-2.3.1-doc/share/doc/nix/manual"
      },
      "image": "arion-base:s089i6gj8kyabwvw06vi7pfahwlp3g0h",
      "ports": ["8000:8000"],
      "sysctls": {},
      "volumes": [
        "/nix/store:/nix/store:ro",
        "/nix/store/cr8cv5d6q6yd0yciw91knljhdpfx4yw6-container-system-env:/run/system:ro"
      ]
    }
  },
  "version": "3.4",
  "x-arion": {
    "images": [
      {
        "image": "/nix/store/s089i6gj8kyabwvw06vi7pfahwlp3g0h-docker-image-arion-base.tar.gz",
        "imageName": "arion-base",
        "imageTag": "s089i6gj8kyabwvw06vi7pfahwlp3g0h"
      }
    ],
    "serviceInfo": { "webserver": { "defaultExec": ["/bin/sh"] } }
  }
}

There could be a mix of x86_64-darwin and x86_64-linux somewhere.

[roberth]

Could you check that arion-pkgs.nix sets the system parameter? This was missing from the documentation arion-pkgs.nix example.

If you use the arion.build or arion.eval functions in Nix, make sure they're from a linux pkgs.

[jbmusso]

Indeed, the system parameter was missing on my setup - I'm unsure how I missed that part.

import <nixpkgs> { system = "x86_64-linux"; }

I confirm that arion's "minimal" example as of yesterday now builds successfully using https://github.com/nix-community/linuxkit-nix as a remote builder.

Output of nix-shell -p nix-info --run "nix-info -m":

- system: `"x86_64-darwin"`
- host os: `Darwin 18.6.0, macOS 10.14.5`
- multi-user?: `no`
- sandbox: `no`
- version: `nix-env (Nix) 2.3.1`
- channels(jb): `"nixpkgs-20.03pre196811.7818f30cc4b, nixos-19.03-19.03.173408.bd6ba87381e"`
- nixpkgs: `/Users/jb/.nix-defexpr/channels/nixpkgs`

Thanks for taking the time to look into this. I'm still getting started in nix and looking forward to contribute back.