UPSTREAM: <carry>: Change docker security opt separator to be compatible with 1.11+

[pmorie]

Closes bug 1413100

[derekwaynecarr]

[merge]

[derekwaynecarr]

re [merge]

[pmorie]

@stevekuznetsov what is the deal with the build here

[stevekuznetsov]

@openshift-bot, the last build failed from the following flakes:

• networking failed on: Networking Test Suite Timed Out #12865

re[merge]

[derekwaynecarr]

@openshift-bot, the last build failed from the following flakes:

• networking failed on: Networking Test Suite Timed Out #12865

re[merge]

[stevekuznetsov]

If that repeats there is some chance it's due to this PR?

[smarterclayton]

Flaked on openshift/origin-gce#12

[pmorie]

@openshift-bot, the last build failed from the following flakes:

networking failed on: #12865

re[merge]

[pmorie]

🎲 🎲

rollin them dice

You've already embarrassed me in front of @derekwaynecarr FOR THE LAST TIME jenkins

[stevekuznetsov]

@pmorie sorry had to abort job, some technical issues. you're still in the queue

[smarterclayton]

Queue went boom, try again

[merge]

[smarterclayton]

Networking hung

[derekwaynecarr]

re [merge] again

[smarterclayton]

This PR is cursed On Feb 9, 2017, at 10:37 PM, OpenShift Bot <notifications@github.com> wrote: continuous-integration/openshift-jenkins/merge FAILURE ( https://ci.openshift.redhat.com/jenkins/job/test_pull_requests_origin_future/63/) (Base Commit: 898266a <898266a> ) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#12831 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABG_p3DnW7QIO-4Pf-oXuUAkOJ8zr2OZks5ra9t2gaJpZM4L4luM> .

[eparis]

oh my god is anything [merge] ing today?

[openshift-bot]

[Test]ing while waiting on the merge queue

[stevekuznetsov]

@openshift-bot, the last build failed from the following flakes:

• networking failed on: Error: Failed to synchronize cache for repo 'updates' #11452

re[merge]

[stevekuznetsov]

@pmorie this really does look like your PR is making networking consistently hang.

[smarterclayton]

Yeah, removing tag this is suspicious

[smarterclayton]

There are lots of things merging and my spider senses are tingling

[smarterclayton]

I wish we had test grid history.

[stevekuznetsov]

@smarterclayton I've been baby-sitting for the queue today and everything else seems legit.

[pmorie]

[testextended][extended:networking-minimal]

Going to try to isolate the networking failure we were seeing.

[pmorie]

Next step for me is to debug the test in question - but from looking at the test code, there should be no deps on this change for that particular test to work/break that wouldn't have broken a ton of other tests. Nonetheless, it does seem that this PR is indicating there's some kind of breakage. So, time to debug.

[pmorie]

I was able to run the failing tests correctly locally, so will need to debug in more detail on the CI environment.

[pmorie]

@stevekuznetsov and I have tracked down the issue here:

1. Docker 1.12 ignores old-format separators
2. Kubernetes 1.5 uses the behaviors of the docker daemon to DTRT wrt SELinux when no SELinux context is provided for the pod, which is the behavior that the origin CI fell into when it was sending the old separator. This is why the jobs ran correctly with SELinux enforcing, even though our separators were being ignored.
3. The networking tests use a DIND setup in order to simulate multiple hosts on a single machine; the DIND version is 1.10, which will fail to start containers if the new-style separator is passed.

@marun is going to submit a change to move the DIND version in the networking tests up to 1.12 today.

[stevekuznetsov]

@marun can we tentatively create a card to move the dind image building to rely on RHEL and pull in the same Docker we use for the AMIs so that there is no drift? Is Fedora a necessary target for those jobs?

[stevekuznetsov]

Prerequisite #12965 has merged -- thanks @marun
let's re[test] this now

[openshift-bot]

Evaluated for origin testextended up to 9f81f6f

[openshift-bot]

Evaluated for origin test up to 9f81f6f

[openshift-bot]

continuous-integration/openshift-jenkins/test SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pull_requests_origin_future/231/) (Base Commit: c4879fc)

[openshift-bot]

continuous-integration/openshift-jenkins/testextended SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pr_origin_extended/1104/) (Base Commit: c4879fc) (Extended Tests: networking-minimal)

[stevekuznetsov]

Newer DIND images are pulling in docker-2:1.12.6-6.gitae7d637.fc25, tests look fine, [merge]

[openshift-bot]

Evaluated for origin merge up to 9f81f6f

[openshift-bot]

continuous-integration/openshift-jenkins/merge SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pull_requests_origin_future/253/) (Base Commit: 2a248fb) (Image: devenv-rhel7_5922)

[jim-minter]

I'm using docker-1.10.3-45.gite03ddb8.fc23.x86_64, and this change means I can't run pods any more:

Error syncing pod, skipping: failed to "StartContainer" for "POD" with RunContainerError: "runContainer: Error response from daemon: Invalid --security-opt: \"label=level:s0:c8,c2\""

Is this intended?

[stevekuznetsov]

Yes. Docker 1.10 is not supported for the current release of Origin.

[jim-minter]

What is the minimum version of Docker now required? 1.12? Where is it documented?

[stevekuznetsov]

The Origin installation documentation lists 1.12 as the minimum version for 1.5

[pweil-]

@pmorie @derekwaynecarr we also required 1.12 for 3.4, this likely needs backported (didn't see another PR from a quick search)