Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pipeline artifact config to the shared config store #2947

Merged

Conversation

afrittoli
Copy link
Member

@afrittoli afrittoli commented Jul 15, 2020

Changes

The pipeline controller loads the artifact storage config maps via
the client on every reconcile. Refactor the artifact storage code
to extract the configuration aspects and move them into the shared
storage.

Update the artifact storage to use config from the store.
Since the artifact storage interface changed as a consequence,
update all the consumers accordingly: input and out resources
and the task run controller.

Depends-on: #2938

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

  • Includes tests (if functionality changed/added)
  • Includes docs (if user facing)
  • Commit messages follow commit message best practices
  • Release notes block has been filled in or deleted (only if no user facing changes)

See the contribution guide for more details.

Double check this list of stuff that's easy to miss:

Reviewer Notes

If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.

Release Notes

NONE

/kind cleanup

@tekton-robot tekton-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note-none Denotes a PR that doesnt merit a release note. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jul 15, 2020
@afrittoli afrittoli force-pushed the feature/2888-pipeline-config branch from 8e9874c to 1f527e9 Compare July 16, 2020 13:22
@afrittoli afrittoli changed the title WIP Add pipeline artifact config to the share store Add pipeline artifact config to the share store Jul 16, 2020
@tekton-robot tekton-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 16, 2020
@afrittoli afrittoli changed the title Add pipeline artifact config to the share store Add pipeline artifact config to the shared config store Jul 16, 2020
@afrittoli afrittoli force-pushed the feature/2888-pipeline-config branch from 1f527e9 to b1f0161 Compare July 16, 2020 14:52
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 82.1% 81.2% -1.0
pkg/artifacts/artifacts_storage.go 80.2% 82.2% 2.0
pkg/reconciler/taskrun/resources/input_resources.go 89.5% 90.9% 1.4
pkg/reconciler/taskrun/resources/output_resource.go 87.5% 89.1% 1.6

@afrittoli afrittoli force-pushed the feature/2888-pipeline-config branch from b1f0161 to 5671706 Compare July 16, 2020 15:22
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 82.1% 81.2% -1.0
pkg/artifacts/artifacts_storage.go 80.2% 82.2% 2.0
pkg/reconciler/pipelinerun/pipelinerun.go 85.9% 86.0% 0.0
pkg/reconciler/taskrun/resources/input_resources.go 89.5% 90.9% 1.4
pkg/reconciler/taskrun/resources/output_resource.go 87.5% 89.1% 1.6

@afrittoli afrittoli force-pushed the feature/2888-pipeline-config branch from 5671706 to 6e22ce7 Compare July 16, 2020 15:53
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 82.1% 81.2% -1.0
pkg/artifacts/artifacts_storage.go 80.2% 82.2% 2.0
pkg/reconciler/pipelinerun/pipelinerun.go 85.9% 86.0% 0.0
pkg/reconciler/taskrun/resources/input_resources.go 89.5% 90.9% 1.4
pkg/reconciler/taskrun/resources/output_resource.go 87.5% 89.1% 1.6

@afrittoli afrittoli force-pushed the feature/2888-pipeline-config branch from 6e22ce7 to b2946a9 Compare July 16, 2020 16:27
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 82.1% 81.2% -1.0
pkg/artifacts/artifacts_storage.go 80.2% 82.2% 2.0
pkg/reconciler/pipelinerun/pipelinerun.go 85.9% 86.0% 0.0
pkg/reconciler/taskrun/resources/input_resources.go 89.5% 90.9% 1.4
pkg/reconciler/taskrun/resources/output_resource.go 87.5% 89.1% 1.6

@afrittoli afrittoli added this to the Pipelines v0.15 milestone Jul 16, 2020
docs/events.md Outdated Show resolved Hide resolved
ServiceAccountFieldName: DefaultBucketServiceFieldName,
}

if location, ok := cfgMap[BucketLocationKey]; ok {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might be able to save some of these checks with the helpers in knative/pkg/configmap:

cm.AsString(BucketLocationKey, &tc.Location)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will check those out, thanks. I could update all config modules together in a separate patch.

pkg/apis/config/artifact_bucket.go Show resolved Hide resolved
@afrittoli afrittoli force-pushed the feature/2888-pipeline-config branch from b2946a9 to ace6640 Compare July 17, 2020 15:27
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 82.1% 81.2% -1.0
pkg/artifacts/artifacts_storage.go 80.2% 82.2% 2.0
pkg/reconciler/taskrun/resources/input_resources.go 89.5% 90.9% 1.4
pkg/reconciler/taskrun/resources/output_resource.go 87.5% 89.1% 1.6

@afrittoli afrittoli force-pushed the feature/2888-pipeline-config branch from ace6640 to 906913c Compare July 17, 2020 15:35
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
internal/builder/v1beta1/pipeline.go 82.1% 81.2% -1.0
pkg/artifacts/artifacts_storage.go 80.2% 82.2% 2.0
pkg/reconciler/pipelinerun/pipelinerun.go 85.9% 86.0% 0.0
pkg/reconciler/taskrun/resources/input_resources.go 89.5% 90.9% 1.4
pkg/reconciler/taskrun/resources/output_resource.go 87.5% 89.1% 1.6

@afrittoli
Copy link
Member Author

/cc @jlpettersson

@tekton-robot tekton-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 25, 2020
@afrittoli afrittoli force-pushed the feature/2888-pipeline-config branch from 906913c to 3c18118 Compare July 27, 2020 14:36
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/artifacts_storage.go 80.2% 82.2% 2.0
pkg/reconciler/taskrun/resources/input_resources.go 89.5% 90.9% 1.4
pkg/reconciler/taskrun/resources/output_resource.go 87.5% 89.1% 1.6

@tekton-robot tekton-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jul 27, 2020
@tekton-robot
Copy link
Collaborator

@afrittoli: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

The pipeline controller loads the artifact storage config maps via
the client on every reconcile. Refactor the artifact storage code
to extract the configuration aspects and move them into the shared
storage.

Update the artifact storage to use config from the store.
Since the artifact storage interface changed as a consequence,
update all the consumers accordingly: input and out resources
and the task run controller.
@afrittoli afrittoli force-pushed the feature/2888-pipeline-config branch from 3c18118 to a99fec2 Compare July 31, 2020 08:23
@tekton-robot
Copy link
Collaborator

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/artifacts/artifacts_storage.go 80.2% 82.2% 2.0
pkg/reconciler/taskrun/resources/input_resources.go 89.5% 90.9% 1.4
pkg/reconciler/taskrun/resources/output_resource.go 87.5% 89.1% 1.6

@vdemeester vdemeester removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 31, 2020
Copy link
Member

@vdemeester vdemeester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Jul 31, 2020
@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sbwsg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 31, 2020
@tekton-robot tekton-robot merged commit 7f71aa8 into tektoncd:master Jul 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm Indicates that a PR is ready to be merged. release-note-none Denotes a PR that doesnt merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants