New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker image volume fix and Sharness tests #2064

Merged
merged 7 commits into from Jan 10, 2016

Conversation

Projects
None yet
5 participants
@chriscool
Contributor

chriscool commented Dec 13, 2015

This PR contains @ChristianKniep's fix from PR #1806 (change Dockerfile order to create VOLUME under user ipfs) and add a Sharness test script (t0300) with some docker tests.
@ChristianKniep's fix is fixing issue #1804 (Boot2Docker: VOLUME prevents chown to have effect).

@jbenet jbenet added the in progress label Dec 13, 2015

@chriscool

This comment has been minimized.

Show comment
Hide comment
@chriscool

chriscool Dec 13, 2015

Contributor

Circle CI is failing with:

expecting success: 
    docker build --rm "$APP_ROOT_DIR" >actual

Post http:///var/run/docker.sock/v1.20/build?cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&memory=0&memswap=0&rm=1&t=&ulimits=null: dial unix /var/run/docker.sock: no such file or directory.
* Are you trying to connect to a TLS-enabled daemon without TLS?
* Is your docker daemon up and running?
not ok 5 - docker image build succeeds

I am not sure it supports docker or maybe some options are needed to make it run docker.

Contributor

chriscool commented Dec 13, 2015

Circle CI is failing with:

expecting success: 
    docker build --rm "$APP_ROOT_DIR" >actual

Post http:///var/run/docker.sock/v1.20/build?cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&memory=0&memswap=0&rm=1&t=&ulimits=null: dial unix /var/run/docker.sock: no such file or directory.
* Are you trying to connect to a TLS-enabled daemon without TLS?
* Is your docker daemon up and running?
not ok 5 - docker image build succeeds

I am not sure it supports docker or maybe some options are needed to make it run docker.

@chriscool

This comment has been minimized.

Show comment
Hide comment
@chriscool

chriscool Dec 13, 2015

Contributor

About CircleCI I found an option in the doc for docker support, so I added a commit to set it.

Contributor

chriscool commented Dec 13, 2015

About CircleCI I found an option in the doc for docker support, so I added a commit to set it.

@chriscool

This comment has been minimized.

Show comment
Hide comment
@chriscool

chriscool Dec 13, 2015

Contributor

Travis-CI supports Docker on Linux only for now.
Let's use a prereq to not run Docker tests on OSX.

Contributor

chriscool commented Dec 13, 2015

Travis-CI supports Docker on Linux only for now.
Let's use a prereq to not run Docker tests on OSX.

@chriscool

This comment has been minimized.

Show comment
Hide comment
@chriscool

chriscool Dec 13, 2015

Contributor

travis-ci/push has this failure in go tests:

14:00:29.390 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:57236: use of closed network connection swarm_listen.go:129

14:00:29.390 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:48892: use of closed network connection swarm_listen.go:129

14:00:29.391 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:46832: use of closed network connection swarm_listen.go:129

14:00:29.395 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:32841: use of closed network connection swarm_listen.go:129

--- FAIL: TestDialWait (0.02s)

    dial_test.go:103: failed to specify addrs: [/ip6/::/tcp/42276]

14:00:29.418 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:38928: use of closed network connection swarm_listen.go:129

--- FAIL: TestDialBackoffClears (0.05s)

    dial_test.go:103: failed to specify addrs: [/ip6/::/tcp/35358]

14:00:29.483 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:54142: use of closed network connection swarm_listen.go:129

14:00:29.490 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:47706: use of closed network connection swarm_listen.go:129 
Contributor

chriscool commented Dec 13, 2015

travis-ci/push has this failure in go tests:

14:00:29.390 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:57236: use of closed network connection swarm_listen.go:129

14:00:29.390 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:48892: use of closed network connection swarm_listen.go:129

14:00:29.391 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:46832: use of closed network connection swarm_listen.go:129

14:00:29.395 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:32841: use of closed network connection swarm_listen.go:129

--- FAIL: TestDialWait (0.02s)

    dial_test.go:103: failed to specify addrs: [/ip6/::/tcp/42276]

14:00:29.418 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:38928: use of closed network connection swarm_listen.go:129

--- FAIL: TestDialBackoffClears (0.05s)

    dial_test.go:103: failed to specify addrs: [/ip6/::/tcp/35358]

14:00:29.483 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:54142: use of closed network connection swarm_listen.go:129

14:00:29.490 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:47706: use of closed network connection swarm_listen.go:129 
@chriscool

This comment has been minimized.

Show comment
Hide comment
@chriscool

chriscool Dec 13, 2015

Contributor

I rebased to remove debug output.

Contributor

chriscool commented Dec 13, 2015

I rebased to remove debug output.

@chriscool

This comment has been minimized.

Show comment
Hide comment
@chriscool

chriscool Dec 13, 2015

Contributor

We get "Unsupported: Exec is not supported by the lxc driver" with CircleCi.

It looks like "docker exec" doesn't work on CircleCi:

https://circleci.com/docs/docker#docker-exec

Contributor

chriscool commented Dec 13, 2015

We get "Unsupported: Exec is not supported by the lxc driver" with CircleCi.

It looks like "docker exec" doesn't work on CircleCi:

https://circleci.com/docs/docker#docker-exec

@chriscool

This comment has been minimized.

Show comment
Hide comment
@chriscool

chriscool Dec 13, 2015

Contributor

I think I will have to do something like this:

exec_docker() {
    if test "$CIRCLECI" = 1
    then
        sudo lxc-attach -n "$(docker inspect --format '{{.Id}}' $1)" -- bash -c "$2"
    else
        docker exec -i "$1" /bin/bash -c "2"
    fi
}
Contributor

chriscool commented Dec 13, 2015

I think I will have to do something like this:

exec_docker() {
    if test "$CIRCLECI" = 1
    then
        sudo lxc-attach -n "$(docker inspect --format '{{.Id}}' $1)" -- bash -c "$2"
    else
        docker exec -i "$1" /bin/bash -c "2"
    fi
}
@chriscool

This comment has been minimized.

Show comment
Hide comment
@chriscool

chriscool Dec 13, 2015

Contributor

There is this error on travis-ci/push:

expecting success: 

        ipfs add -r --hidden mountdir/planets >actual



ok 12 - 'ipfs add -r --hidden' succeeds

expecting success: 

        cat >expected <<-\EOF &&

            added QmcAREBcjgnUpKfyFmUGnfajA1NQS5ydqRp7WfqZ6JF8Dx planets/.asteroids/ceres.txt

            added QmZ5eaLybJ5GUZBNwy24AA9EEDTDpA4B8qXnuN3cGxu2uF planets/.asteroids/pallas.txt

            added Qmf6rbs5GF85anDuoxpSAdtuZPM9D2Yt3HngzjUVSQ7kDV planets/.asteroids

            added QmaowqjedBkUrMUXgzt9c2ZnAJncM9jpJtkFfgdFstGr5a planets/.charon.txt

            added QmU4zFD5eJtRBsWC63AvpozM9Atiadg9kPVTuTrnCYJiNF planets/.pluto.txt

            added QmZy3khu7qf696i5HtkgL2NotsCZ8wzvNZJ1eUdA5n8KaV planets/mars.txt

            added QmQnv4m3Q5512zgVtpbJ9z85osQrzZzGRn934AGh6iVEXz planets/venus.txt

            added QmetajtFdmzhWYodAsZoVZSiqpeJDAiaw2NwbM3xcWcpDj planets

        EOF

        test_cmp expected actual



> diff -u expected actual

--- expected    2015-12-13 14:14:43.270334999 +0000

+++ actual  2015-12-13 14:14:43.258334999 +0000

@@ -1,8 +1,8 @@

-added QmcAREBcjgnUpKfyFmUGnfajA1NQS5ydqRp7WfqZ6JF8Dx planets/.asteroids/ceres.txt

-added QmZ5eaLybJ5GUZBNwy24AA9EEDTDpA4B8qXnuN3cGxu2uF planets/.asteroids/pallas.txt

-added Qmf6rbs5GF85anDuoxpSAdtuZPM9D2Yt3HngzjUVSQ7kDV planets/.asteroids

+added QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH planets/.asteroids/ceres.txt

+added QmTF2HDVTJMXWGUkrtAFfQpYxom5A8Tj7S38SzPfGVCD5P planets/.asteroids/pallas.txt

+added Qmed5hbZhHpbTEmmTGiVStW6oMmZqDorNkMwV4aWWbbWPf planets/.asteroids

 added QmaowqjedBkUrMUXgzt9c2ZnAJncM9jpJtkFfgdFstGr5a planets/.charon.txt

 added QmU4zFD5eJtRBsWC63AvpozM9Atiadg9kPVTuTrnCYJiNF planets/.pluto.txt

 added QmZy3khu7qf696i5HtkgL2NotsCZ8wzvNZJ1eUdA5n8KaV planets/mars.txt

 added QmQnv4m3Q5512zgVtpbJ9z85osQrzZzGRn934AGh6iVEXz planets/venus.txt

-added QmetajtFdmzhWYodAsZoVZSiqpeJDAiaw2NwbM3xcWcpDj planets

+added Qma69eGiYgSFiuzUvM5yPDsXyvmGVgaryQE84qYnehCk2r planets

not ok 13 - 'ipfs add -r --hidden' did include . files
Contributor

chriscool commented Dec 13, 2015

There is this error on travis-ci/push:

expecting success: 

        ipfs add -r --hidden mountdir/planets >actual



ok 12 - 'ipfs add -r --hidden' succeeds

expecting success: 

        cat >expected <<-\EOF &&

            added QmcAREBcjgnUpKfyFmUGnfajA1NQS5ydqRp7WfqZ6JF8Dx planets/.asteroids/ceres.txt

            added QmZ5eaLybJ5GUZBNwy24AA9EEDTDpA4B8qXnuN3cGxu2uF planets/.asteroids/pallas.txt

            added Qmf6rbs5GF85anDuoxpSAdtuZPM9D2Yt3HngzjUVSQ7kDV planets/.asteroids

            added QmaowqjedBkUrMUXgzt9c2ZnAJncM9jpJtkFfgdFstGr5a planets/.charon.txt

            added QmU4zFD5eJtRBsWC63AvpozM9Atiadg9kPVTuTrnCYJiNF planets/.pluto.txt

            added QmZy3khu7qf696i5HtkgL2NotsCZ8wzvNZJ1eUdA5n8KaV planets/mars.txt

            added QmQnv4m3Q5512zgVtpbJ9z85osQrzZzGRn934AGh6iVEXz planets/venus.txt

            added QmetajtFdmzhWYodAsZoVZSiqpeJDAiaw2NwbM3xcWcpDj planets

        EOF

        test_cmp expected actual



> diff -u expected actual

--- expected    2015-12-13 14:14:43.270334999 +0000

+++ actual  2015-12-13 14:14:43.258334999 +0000

@@ -1,8 +1,8 @@

-added QmcAREBcjgnUpKfyFmUGnfajA1NQS5ydqRp7WfqZ6JF8Dx planets/.asteroids/ceres.txt

-added QmZ5eaLybJ5GUZBNwy24AA9EEDTDpA4B8qXnuN3cGxu2uF planets/.asteroids/pallas.txt

-added Qmf6rbs5GF85anDuoxpSAdtuZPM9D2Yt3HngzjUVSQ7kDV planets/.asteroids

+added QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH planets/.asteroids/ceres.txt

+added QmTF2HDVTJMXWGUkrtAFfQpYxom5A8Tj7S38SzPfGVCD5P planets/.asteroids/pallas.txt

+added Qmed5hbZhHpbTEmmTGiVStW6oMmZqDorNkMwV4aWWbbWPf planets/.asteroids

 added QmaowqjedBkUrMUXgzt9c2ZnAJncM9jpJtkFfgdFstGr5a planets/.charon.txt

 added QmU4zFD5eJtRBsWC63AvpozM9Atiadg9kPVTuTrnCYJiNF planets/.pluto.txt

 added QmZy3khu7qf696i5HtkgL2NotsCZ8wzvNZJ1eUdA5n8KaV planets/mars.txt

 added QmQnv4m3Q5512zgVtpbJ9z85osQrzZzGRn934AGh6iVEXz planets/venus.txt

-added QmetajtFdmzhWYodAsZoVZSiqpeJDAiaw2NwbM3xcWcpDj planets

+added Qma69eGiYgSFiuzUvM5yPDsXyvmGVgaryQE84qYnehCk2r planets

not ok 13 - 'ipfs add -r --hidden' did include . files
@chriscool

This comment has been minimized.

Show comment
Hide comment
@chriscool

chriscool Dec 13, 2015

Contributor

On travis-ci/pr there is this error:

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.

The build has been terminated
Contributor

chriscool commented Dec 13, 2015

On travis-ci/pr there is this error:

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.

The build has been terminated
@jbenet

This comment has been minimized.

Show comment
Hide comment
@jbenet

jbenet Dec 13, 2015

Member

Travis-CI supports Docker on Linux only for now.
Let's use a prereq to not run Docker tests on OSX.

ok

I think I will have to do something like this:

exec_docker()

wow, bummer. maybe put that in a shell scrpt and call it docker-exec, we can call that everywhere we want to do docker exec.

There is this error on travis-ci/push:

the hashes seem different. content somehow different??

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
The build has been terminated

ugh yeah that sometimes happens. there's some test that hangs travis-- we should look into them. unrelated i think

Member

jbenet commented Dec 13, 2015

Travis-CI supports Docker on Linux only for now.
Let's use a prereq to not run Docker tests on OSX.

ok

I think I will have to do something like this:

exec_docker()

wow, bummer. maybe put that in a shell scrpt and call it docker-exec, we can call that everywhere we want to do docker exec.

There is this error on travis-ci/push:

the hashes seem different. content somehow different??

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
The build has been terminated

ugh yeah that sometimes happens. there's some test that hangs travis-- we should look into them. unrelated i think

@jbenet

This comment has been minimized.

Show comment
Hide comment
@jbenet

jbenet Dec 13, 2015

Member

the closed network connection stuff is odd. probably some issue with the docker networking

Member

jbenet commented Dec 13, 2015

the closed network connection stuff is odd. probably some issue with the docker networking

@chriscool chriscool added the RFM label Dec 15, 2015

@chriscool

This comment has been minimized.

Show comment
Hide comment
@chriscool

chriscool Dec 15, 2015

Contributor

Yeah, with the docker_exec() that has a special case for CircleCI it works on CircleCI!
The other build failures are the usual ones on Travis OSX and in the go tests.

Contributor

chriscool commented Dec 15, 2015

Yeah, with the docker_exec() that has a special case for CircleCI it works on CircleCI!
The other build failures are the usual ones on Travis OSX and in the go tests.

@chriscool chriscool referenced this pull request Dec 15, 2015

Closed

Sprint Dec 14 #74

@jbenet

This comment has been minimized.

Show comment
Hide comment
@jbenet

jbenet Dec 16, 2015

Member

@chriscool i think the go test failure here is not a usual one-- i do not think i have seen it on travis without docker.

Member

jbenet commented Dec 16, 2015

@chriscool i think the go test failure here is not a usual one-- i do not think i have seen it on travis without docker.

@chriscool

This comment has been minimized.

Show comment
Hide comment
@chriscool

chriscool Dec 16, 2015

Contributor

The got test errors are:

22:15:58.314 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:48529: use of closed network connection swarm_listen.go:129

22:15:58.319 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:44342: use of closed network connection swarm_listen.go:129

--- FAIL: TestDialWait (0.02s)

    dial_test.go:103: failed to specify addrs: [/ip6/::/tcp/32781]

--- FAIL: TestDialBackoffClears (0.03s)

    dial_test.go:103: failed to specify addrs: [/ip6/::/tcp/41355]

Yeah, they are perhaps related to the Travis CI environment with docker.

@whyrusleeping do you know what in the Travis CI environment with docker could cause those failures?

Contributor

chriscool commented Dec 16, 2015

The got test errors are:

22:15:58.314 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:48529: use of closed network connection swarm_listen.go:129

22:15:58.319 ERROR     swarm2: swarm listener accept error: peerstream listener failed: accept tcp 127.0.0.1:44342: use of closed network connection swarm_listen.go:129

--- FAIL: TestDialWait (0.02s)

    dial_test.go:103: failed to specify addrs: [/ip6/::/tcp/32781]

--- FAIL: TestDialBackoffClears (0.03s)

    dial_test.go:103: failed to specify addrs: [/ip6/::/tcp/41355]

Yeah, they are perhaps related to the Travis CI environment with docker.

@whyrusleeping do you know what in the Travis CI environment with docker could cause those failures?

@whyrusleeping

This comment has been minimized.

Show comment
Hide comment
@whyrusleeping

whyrusleeping Jan 10, 2016

Member

@chriscool i've got the ipv6 stuff fixed now, but it appears that travis and t0042 don't get along well...

Member

whyrusleeping commented Jan 10, 2016

@chriscool i've got the ipv6 stuff fixed now, but it appears that travis and t0042 don't get along well...

@chriscool

This comment has been minimized.

Show comment
Hide comment
@chriscool

chriscool Jan 10, 2016

Contributor

@whyrusleeping I think the Travis tests passed except the usual No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself errors.

Contributor

chriscool commented Jan 10, 2016

@whyrusleeping I think the Travis tests passed except the usual No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself errors.

@chriscool

This comment has been minimized.

Show comment
Hide comment
@chriscool

chriscool Jan 10, 2016

Contributor

And one of those failures is on Linux not OSX!
See: https://travis-ci.org/ipfs/go-ipfs/jobs/101416974

Contributor

chriscool commented Jan 10, 2016

And one of those failures is on Linux not OSX!
See: https://travis-ci.org/ipfs/go-ipfs/jobs/101416974

@chriscool

This comment has been minimized.

Show comment
Hide comment
@chriscool

chriscool Jan 10, 2016

Contributor

It's the one from continuous-integration/travis-ci/pr.

Contributor

chriscool commented Jan 10, 2016

It's the one from continuous-integration/travis-ci/pr.

@whyrusleeping

This comment has been minimized.

Show comment
Hide comment
Member

whyrusleeping commented Jan 10, 2016

LGTM

ChristianKniep and others added some commits Oct 6, 2015

change Dockerfile order to create VOLUME under user ipfs
License: MIT
Signed-off-by: Christian Kniep <christian@qnib.org>
sharness: add t0300-docker-image.sh
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Travis-CI: add docker service
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
CircleCI: add docker service
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Add and use a DOCKER prereq
License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
test/ipfs-test-lib: add docker support
We have to do something special for CircleCI in docker_exec()
because "docker exec" doesn't work on CircleCi:

https://circleci.com/docs/docker#docker-exec

We indeed get "Unsupported: Exec is not supported by the lxc
driver" with CircleCi, when using "docker exec".

License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
@chriscool

This comment has been minimized.

Show comment
Hide comment
@chriscool

chriscool Jan 10, 2016

Contributor

Rebased on current master. Let's have a look at the tests.

Contributor

chriscool commented Jan 10, 2016

Rebased on current master. Let's have a look at the tests.

force use of ipv4 in test
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

whyrusleeping added a commit that referenced this pull request Jan 10, 2016

Merge pull request #2064 from ipfs/ipfs-volume
Docker image volume fix and Sharness tests

@whyrusleeping whyrusleeping merged commit dc4f5af into master Jan 10, 2016

2 of 5 checks passed

continuous-integration/travis-ci/push The Travis CI build could not complete due to an error
Details
continuous-integration/appveyor/branch AppVeyor build failed
Details
continuous-integration/appveyor/pr AppVeyor build failed
Details
ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@whyrusleeping whyrusleeping deleted the ipfs-volume branch Jan 10, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment