ci-operator: stop allowing users to set artifact dir

[stevekuznetsov]

No longer are users able to set the explicit directory for:

• ci-operator outputs
• simple Pod tests
• multi-stage test steps

ci-operator itself will read the ${ARTIFACTS} value to determine
where to put its own artifacts (like the output jUnit files). This is in
effect what already happens, as this is the fallback to when the flag is
not passed, and we do not pass it in generated jobs. The flag will
remain in a deprecated manner so as to not break users.

All test steps except for Template tests are already using the
test-infra pod-utils to upload, and therefore are not allowed to set a
non-standard directory, anyway. Nothing changes for these steps. It is
now not possible to opt out of this by telling ci-operator not to use
pod-utils.

We continue to use the deprecated artifact fetching sidecar code for
Templates, but only for templates.

This will cause a breaking change to prowgen as our previous automation
explicitly wrote our defaulted value into all configs even if they never
set it.

Signed-off-by: Steve Kuznetsov skuznets@redhat.com

/assign @alvaroaleman @petr-muller @bbguimaraes

[stevekuznetsov]

Part of DPTP-1668

[stevekuznetsov]

This is suspicious but I think templates don't actually use this, they use addArtifactsToPod ...

[stevekuznetsov]

Hmm I think I can split this up

[stevekuznetsov]

OK, so it theoretically should be possible to split this into steps:

1. remove the feature from basic container jobs
2. remove it from step registry stuff
3. remove it from the ci-operator top-level

I'll try to do that - there are areas where we infer 1&2 from 3 randomly so it's a bit messy but probably doable.

[stevekuznetsov]

@alvaroaleman @petr-muller updated this to be minimal now that the rest of the changes have come in. Notably, this removes the --artifact-dir setting from ci-operator itself. I think this is a useful thing to do as:

1. we never set it explicitly
2. we only care about using it when Prow's running us

In openshift/release today, we have this:

$ git grep -h artifact-dir | head -n 1
        - --artifact-dir=$(ARTIFACTS)
$ git grep artifact-dir | wc -l
563
$ git grep artifact-dir | grep -v ARTIFACTS | wc -l
0

So there's 563 places where we set it to what would otherwise be the default value anyway, so there's zero utility to this flag. In addition, having to pass this directory around everywhere in our codebase ended up being a bad cargo-cult ... half the places didn't even end up using it, and in some cases like multi-stage it looks like we accidentally used this instead of the per-step one ...

[stevekuznetsov]

/retest

[stevekuznetsov]

@alvaroaleman updated

[stevekuznetsov]

Seeing a lot of this lately

 error: error creating buildah builder: Error writing blob: error storing blob to file "/var/tmp/storage350157992/5": unexpected EOF 

/retest

[stevekuznetsov]

Rebased and ready @alvaroaleman @petr-muller

[petr-muller]

LGTM

/hold
Holding to allow resolving Alvaro's comment

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: petr-muller, stevekuznetsov

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [petr-muller,stevekuznetsov]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[stevekuznetsov]

/hold cancel

You need to opt in via flag