Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Surface artifacts through sidecar container logs. #7883

Merged
merged 1 commit into from May 9, 2024
Merged

feat: Surface artifacts through sidecar container logs. #7883

merged 1 commit into from May 9, 2024

Conversation

ericzzzzzzz
Copy link
Contributor

@ericzzzzzzz ericzzzzzzz commented Apr 15, 2024 •
edited by JeromeJu

Changes

  • This Pr uses the similar approach as TEP-0142: Surface step results via sidecar logs #7414 to surface large artifacts to TaskRun.status.steps.
  • The flag result-from:sidecar-logs is used to guard this feature, it is also the same as surfacing step results through sidecar logs.
  • As we don't have a API to allow users to declare if a step is going to produce artifacts, we check if a step env, command, args, script contains artifacts path to determine if we should create an artifact sidecar container should be created for a task.

Fixes: #7869

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

  • Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
  • Has Tests included if any functionality added or changed
  • pre-commit Passed
  • Follows the commit message standard
  • Meets the Tekton contributor standards (including functionality, content, code)
  • Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
  • Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings). See some examples of good release notes.
  • Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

Surface artifacts through sidecar container logs.

@tekton-robot
Copy link
Collaborator

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@tekton-robot tekton-robot added release-note-none Denotes a PR that doesnt merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels Apr 15, 2024
@tekton-robot tekton-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Apr 15, 2024
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
cmd/sidecarlogartifacts/main.go Do not exist 0.0%
pkg/apis/pipeline/images.go 100.0% 87.5% -12.5
pkg/apis/pipeline/v1/task_validation.go 98.6% 98.3% -0.3
pkg/entrypoint/entrypointer.go 86.8% 86.9% 0.0
pkg/pod/entrypoint.go 94.1% 93.2% -0.9
pkg/pod/pod.go 93.6% 92.6% -1.0
pkg/pod/status.go 94.7% 94.2% -0.5

@ericzzzzzzz
Copy link
Contributor Author

/kind feat

@tekton-robot
Copy link
Collaborator

@ericzzzzzzz: The label(s) kind/feat cannot be applied, because the repository doesn't have them.

In response to this:

/kind feat

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ericzzzzzzz
Copy link
Contributor Author

/kind feature

@tekton-robot tekton-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Apr 22, 2024
@ericzzzzzzz ericzzzzzzz marked this pull request as ready for review April 22, 2024 14:44
@tekton-robot tekton-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 22, 2024
@tekton-robot tekton-robot requested a review from jerop April 22, 2024 14:44
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
cmd/sidecarlogartifacts/main.go Do not exist 0.0%
pkg/apis/pipeline/images.go 100.0% 87.5% -12.5
pkg/apis/pipeline/v1/task_validation.go 98.6% 98.3% -0.3
pkg/entrypoint/entrypointer.go 86.8% 86.9% 0.0
pkg/pod/entrypoint.go 94.1% 93.2% -0.9
pkg/pod/pod.go 93.6% 92.6% -1.0
pkg/pod/status.go 94.7% 94.2% -0.5

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
cmd/sidecarlogartifacts/main.go Do not exist 0.0%
internal/sidecarlogartifacts/sidecarlogartifacts.go Do not exist 65.1%
pkg/apis/pipeline/images.go 100.0% 87.5% -12.5
pkg/apis/pipeline/v1/task_validation.go 98.6% 98.3% -0.3
pkg/entrypoint/entrypointer.go 86.8% 86.9% 0.0
pkg/pod/entrypoint.go 94.1% 93.2% -0.9
pkg/pod/pod.go 93.6% 92.6% -1.0
pkg/pod/status.go 94.7% 94.2% -0.5

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
cmd/sidecarlogartifacts/main.go Do not exist 0.0%
internal/sidecarlogartifacts/sidecarlogartifacts.go Do not exist 65.1%
pkg/apis/pipeline/images.go 100.0% 87.5% -12.5
pkg/apis/pipeline/v1/task_validation.go 98.6% 98.3% -0.3
pkg/entrypoint/entrypointer.go 86.8% 86.9% 0.0
pkg/pod/entrypoint.go 94.1% 93.2% -0.9
pkg/pod/pod.go 93.6% 92.6% -1.0
pkg/pod/status.go 94.7% 94.2% -0.5

@ericzzzzzzz ericzzzzzzz force-pushed the surface-artifacts-through-sidecar branch from d39a81f to 6019951 Compare April 22, 2024 17:16
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
cmd/sidecarlogartifacts/main.go Do not exist 0.0%
internal/sidecarlogartifacts/sidecarlogartifacts.go Do not exist 65.1%
pkg/apis/pipeline/images.go 100.0% 87.5% -12.5
pkg/apis/pipeline/v1/task_validation.go 98.6% 98.3% -0.3
pkg/entrypoint/entrypointer.go 86.8% 86.9% 0.0
pkg/pod/entrypoint.go 94.1% 93.2% -0.9
pkg/pod/pod.go 93.6% 92.6% -1.0
pkg/pod/status.go 94.7% 94.2% -0.5

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
cmd/sidecarlogartifacts/main.go Do not exist 0.0%
internal/sidecarlogartifacts/sidecarlogartifacts.go Do not exist 65.1%
pkg/apis/pipeline/images.go 100.0% 87.5% -12.5
pkg/apis/pipeline/v1/task_validation.go 98.6% 98.3% -0.3
pkg/entrypoint/entrypointer.go 86.8% 86.9% 0.0
pkg/pod/entrypoint.go 94.1% 93.2% -0.9
pkg/pod/pod.go 93.6% 92.6% -1.0
pkg/pod/status.go 94.7% 94.2% -0.5

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
cmd/sidecarlogartifacts/main.go Do not exist 0.0%
internal/sidecarlogartifacts/sidecarlogartifacts.go Do not exist 65.1%
pkg/apis/pipeline/v1/task_validation.go 98.6% 98.3% -0.3
pkg/entrypoint/entrypointer.go 86.8% 86.9% 0.0
pkg/pod/entrypoint.go 94.1% 93.2% -0.9
pkg/pod/pod.go 93.6% 93.7% 0.1
pkg/pod/status.go 94.7% 94.2% -0.5

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
cmd/sidecarlogartifacts/main.go Do not exist 0.0%
internal/sidecarlogartifacts/sidecarlogartifacts.go Do not exist 65.1%
pkg/apis/pipeline/v1/task_validation.go 98.6% 98.3% -0.3
pkg/entrypoint/entrypointer.go 86.8% 86.9% 0.0
pkg/pod/entrypoint.go 94.1% 93.2% -0.9
pkg/pod/pod.go 93.6% 93.7% 0.1
pkg/pod/status.go 94.7% 94.2% -0.5

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
cmd/sidecarlogartifacts/main.go Do not exist 0.0%
internal/sidecarlogartifacts/sidecarlogartifacts.go Do not exist 67.2%
pkg/entrypoint/entrypointer.go 86.8% 86.9% 0.0
pkg/pod/entrypoint.go 94.1% 94.2% 0.1
pkg/pod/pod.go 93.6% 93.7% 0.1
pkg/pod/status.go 94.7% 94.0% -0.7

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
cmd/sidecarlogartifacts/main.go Do not exist 0.0%
internal/sidecarlogartifacts/sidecarlogartifacts.go Do not exist 67.2%
pkg/entrypoint/entrypointer.go 86.8% 86.9% 0.0
pkg/pod/entrypoint.go 94.1% 94.2% 0.1
pkg/pod/pod.go 93.6% 93.7% 0.1
pkg/pod/status.go 94.7% 94.0% -0.7

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
cmd/sidecarlogartifacts/main.go Do not exist 0.0%
internal/sidecarlogartifacts/sidecarlogartifacts.go Do not exist 86.4%
pkg/pod/entrypoint.go 94.1% 94.2% 0.1
pkg/pod/pod.go 93.6% 93.7% 0.1
pkg/pod/status.go 94.7% 94.0% -0.7

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
cmd/sidecarlogartifacts/main.go Do not exist 0.0%
internal/sidecarlogartifacts/sidecarlogartifacts.go Do not exist 86.4%
pkg/pod/entrypoint.go 94.1% 94.2% 0.1
pkg/pod/pod.go 93.6% 93.7% 0.1
pkg/pod/status.go 94.7% 94.0% -0.7

@ericzzzzzzz ericzzzzzzz force-pushed the surface-artifacts-through-sidecar branch from c2c1c22 to 8efcd61 Compare April 22, 2024 19:51
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
cmd/sidecarlogartifacts/main.go Do not exist 0.0%
internal/sidecarlogartifacts/sidecarlogartifacts.go Do not exist 86.4%
pkg/pod/entrypoint.go 94.1% 94.2% 0.1
pkg/pod/pod.go 93.6% 93.7% 0.1
pkg/pod/status.go 94.7% 94.0% -0.7

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
cmd/sidecarlogartifacts/main.go Do not exist 0.0%
internal/sidecarlogartifacts/sidecarlogartifacts.go Do not exist 86.4%
pkg/pod/entrypoint.go 94.1% 94.2% 0.1
pkg/pod/pod.go 93.6% 93.7% 0.1
pkg/pod/status.go 94.7% 94.0% -0.7

@chitrangpatel chitrangpatel added this to the Pipeline v0.60 milestone Apr 29, 2024
@chitrangpatel
Copy link
Member

Thanks for the changes. I would love to see a test where we extract both results and artifacts from sidecar logs so that we can validate that they both work at the same time.

@chitrangpatel
Copy link
Member

Otherwise lgtm

@ericzzzzzzz ericzzzzzzz force-pushed the surface-artifacts-through-sidecar branch from bc59e03 to 1b04d6f Compare April 30, 2024 17:00
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/sidecarlogresults/sidecarlogresults.go 89.6% 87.2% -2.3
pkg/pod/pod.go 93.6% 93.8% 0.2
pkg/pod/status.go 94.7% 92.9% -1.8

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
internal/sidecarlogresults/sidecarlogresults.go 89.6% 87.2% -2.3
pkg/pod/pod.go 93.6% 93.8% 0.2
pkg/pod/status.go 94.7% 92.9% -1.8

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 2, 2024
@JeromeJu JeromeJu self-assigned this May 6, 2024
@ericzzzzzzz ericzzzzzzz force-pushed the surface-artifacts-through-sidecar branch from 1b04d6f to b86b023 Compare May 6, 2024 18:55
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
internal/sidecarlogresults/sidecarlogresults.go 89.6% 87.2% -2.3
pkg/pod/pod.go 93.6% 93.8% 0.2
pkg/pod/status.go 94.7% 93.4% -1.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/sidecarlogresults/sidecarlogresults.go 89.6% 87.2% -2.3
pkg/pod/pod.go 93.6% 93.8% 0.2
pkg/pod/status.go 94.7% 93.4% -1.3

JeromeJu
JeromeJu reviewed May 6, 2024
View reviewed changes
Copy link
Member

@JeromeJu JeromeJu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ericzzzzzzz ! This generally looks good. Just some nit picking and do we have any failure test cases for the coverage?

internal/sidecarlogresults/sidecarlogresults.go Outdated Show resolved Hide resolved
pkg/entrypoint/entrypointer.go Outdated Show resolved Hide resolved
pkg/entrypoint/entrypointer.go Outdated Show resolved Hide resolved
pkg/pod/pod.go Outdated Show resolved Hide resolved
@ericzzzzzzz ericzzzzzzz force-pushed the surface-artifacts-through-sidecar branch from b86b023 to e684c6d Compare May 7, 2024 15:03
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/sidecarlogresults/sidecarlogresults.go 89.6% 87.2% -2.3
pkg/pod/pod.go 93.6% 93.9% 0.3
pkg/pod/status.go 94.7% 93.4% -1.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
internal/sidecarlogresults/sidecarlogresults.go 89.6% 87.2% -2.3
pkg/pod/pod.go 93.6% 93.9% 0.3
pkg/pod/status.go 94.7% 93.4% -1.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/sidecarlogresults/sidecarlogresults.go 89.6% 87.9% -1.6
pkg/pod/pod.go 93.6% 93.9% 0.3
pkg/pod/status.go 94.7% 93.4% -1.3

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
internal/sidecarlogresults/sidecarlogresults.go 89.6% 87.9% -1.6
pkg/pod/pod.go 93.6% 93.9% 0.3
pkg/pod/status.go 94.7% 93.4% -1.3

@ericzzzzzzz ericzzzzzzz force-pushed the surface-artifacts-through-sidecar branch from dc6c3f8 to 6c9e9b5 Compare May 7, 2024 17:01
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/sidecarlogresults/sidecarlogresults.go 89.6% 87.9% -1.6
pkg/pod/pod.go 93.6% 93.9% 0.3
pkg/pod/status.go 94.7% 93.6% -1.1

@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File Old Coverage New Coverage Delta
internal/sidecarlogresults/sidecarlogresults.go 89.6% 87.9% -1.6
pkg/pod/pod.go 93.6% 93.9% 0.3
pkg/pod/status.go 94.7% 93.6% -1.1

@ericzzzzzzz
Copy link
Contributor Author

Thanks @ericzzzzzzz ! This generally looks good. Just some nit picking and do we have any failure test cases for the coverage?

Thanks for the review! I added some tests, the some failure cases are related to file operations and we don't have established pattern to mock those, others are related to json marshal operation, it shouldn't happen as we are passing a struct with all tagged public fields

@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chitrangpatel, JeromeJu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [JeromeJu,chitrangpatel]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@JeromeJu
Copy link
Member

JeromeJu commented May 9, 2024

Thanks @ericzzzzzzz ! This generally looks good. Just some nit picking and do we have any failure test cases for the coverage?

Thanks for the review! I added some tests, the some failure cases are related to file operations and we don't have established pattern to mock those, others are related to json marshal operation, it shouldn't happen as we are passing a struct with all tagged public fields

Understood - then maybe wrapping the errors with the appropriate context might be the way to go.

@chitrangpatel
Copy link
Member

I'm merging this PR since it is basically taking the exact same approach as for larger results to extract artifacts. This has been open for a while and blocking other work. The same functionality has already been implemented for surfacing the information via TerminationMessages. There is nothing novel here.

/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label May 9, 2024
@tekton-robot tekton-robot merged commit af285dd into tektoncd:main May 9, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@chitrangpatel chitrangpatel chitrangpatel approved these changes

@JeromeJu JeromeJu JeromeJu approved these changes

@afrittoli afrittoli Awaiting requested review from afrittoli

@vdemeester vdemeester Awaiting requested review from vdemeester

@jerop jerop Awaiting requested review from jerop

Assignees

@chitrangpatel chitrangpatel

@JeromeJu JeromeJu

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/feature Categorizes issue or PR as related to a new feature. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Milestone
Pipeline v0.60
Development

Successfully merging this pull request may close these issues.

TEP-0147 surface artifacts through sidecar log
4 participants
@ericzzzzzzz @tekton-robot @chitrangpatel @JeromeJu