Run a prowjob with podutils locally

[BenTheElder]

Once we break out git checkout etc from the job container to being integrated with prow we can map kubernetes prow jobs to a wrapper around docker that translates the pod spec and mounts through the local git checkout and skips the log upload.

From a discussion with @dashpole and @krzyzacy

[BenTheElder]

cc @cjwagner @stevekuznetsov

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[BenTheElder]

/remove-lifecycle stale
this is becoming more feasible soon.

[BenTheElder]

/assign
/priority backlog

[stevekuznetsov]

/assign

[stevekuznetsov]

Talked with Ben on Slack, we want a command line tool that will take as input:

• Prow config (infra file + job dir)
• job name
• local directories holding code under test

This tool should spit out a docker run command for the user to use (or do we want to use the Docker client library to just spawn the container and tail logs?)

/cc @BenTheElder

[BenTheElder]

@stevekuznetsov even just exec'ing to docker is probably fine, with a --dry-run mode to print instead?

The client libraries so far haven't really been more stable than the CLI, and being able to print out the exec call is helpful for debugging. That said the client libs are already vendored for barnacle since it can rely on the docker version underneath it.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[JoelSpeed]

Is this the kind of project that would benefit from a KEP?

[krzyzacy]

(Renamed the issue)

cc @munnerz @fejta

[figo]

@krzyzacy @BenTheElder i believe i am doing something really close to what the PR trying to achieve.
develop and debug Prow job in my laptop (run minikube/KIND cluster in my laptop), debugging a Prow job locally is becoming:
make build # build the ci job container (has Prow podutils buildin)
kubectl create -f job.yaml deploy the ci job to local cluster

i am thinking to write a KEP, so that we could have more discussion there, any suggestion?

[BenTheElder]

@figo I'm not sure if this needs a KEP, but that's really cool : D

[stevekuznetsov]

@figo agreed with @BenTheElder here -- no need for a KEP, if you want to write up something quick or post a WIP PR we can talk about it in the SIG meeting and you could just contribute it to the repo :)

[fejta]

Here's the design I would like to see: https://docs.google.com/document/d/1O9edBOAU6D_Wqq-3lFI-si2lEAJwgrEbpwU9dgDjII4/edit?usp=sharing

This does not use kind, instead it translates the prowjob into the equivalent docker container and uses the command line to run it.

/assign

@figo how are things going on your end?

[figo]

Here's the design I would like to see: https://docs.google.com/document/d/1O9edBOAU6D_Wqq-3lFI-si2lEAJwgrEbpwU9dgDjII4/edit?usp=sharing

This does not use kind, instead it translates the prowjob into the equivalent docker container and uses the command line to run it.

/assign

@figo how are things going on your end?

i have not writing anything yet beside i am practicing it with KIND approach, your design doc is good place for discussions, i will join there.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[cjwagner]

Duplicate of #13475
Resolved by #13561

[cjwagner]

/close

[k8s-ci-robot]

@cjwagner: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.