Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure to load image watcher config #1251

Merged
merged 5 commits into from Dec 15, 2020
Merged

Ensure to load image watcher config #1251

merged 5 commits into from Dec 15, 2020

Conversation

nakabonne
Copy link
Member

What this PR does / why we need it:
This PR allows piped to load image watcher files after filtering those based on includes and excludes.

Which issue(s) this PR fixes:

Fixes #1211

Does this PR introduce a user-facing change?:

NONE

@pipecd-bot
Copy link
Collaborator

COVERAGE

Code coverage for golang is 33.83%. This pull request increases coverage by 0.29%.

File Function Base Head Diff
pkg/config/image_watcher.go filterImageWatcherFiles -- 100.00% +100.00%
pkg/config/piped.go PipedImageWatcher.Validate -- 100.00% +100.00%
pkg/model/deployment.go DeploymentStatusesFromStrings -- 0.00% +0.00%
pkg/model/deployment.go DeploymentStatusStrings -- 0.00% +0.00%
pkg/app/piped/logpersister/stagelogpersister.go stageLogPersister.flushFromLastCheckpoint 50.00% 61.11% +11.11%
pkg/config/image_watcher.go LoadImageWatcher 0.00% 0.00% +0.00%
pkg/config/piped.go PipedSpec.Validate 56.25% 55.56% -0.69%
pkg/app/pipectl/cmd/application/add.go newAddCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/add.go add.run 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/application.go NewCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/sync.go newSyncCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/sync.go sync.run 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/sync.go makeStatuses 100.00% -- -100.00%
pkg/app/pipectl/cmd/application/sync.go availableStatuses 100.00% -- -100.00%
pkg/app/pipectl/cmd/deployment/deployment.go NewCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/deployment/waitstatus.go newWaitStatusCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/deployment/waitstatus.go waitStatus.run 0.00% -- +-0.00%
pkg/app/pipectl/cmd/deployment/waitstatus.go makeStatuses 100.00% -- -100.00%
pkg/app/pipectl/cmd/deployment/waitstatus.go availableStatuses 100.00% -- -100.00%

pkg/app/piped/imagewatcher/watcher.go Outdated Show resolved Hide resolved
repos[repo.RepoID] = struct{}{}

if len(repo.Includes) != 0 && len(repo.Excludes) != 0 {
return fmt.Errorf("both includes and excludes are given in %s", repo.RepoID)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have any reasons for not allowing this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our recognition might be different. I'm thinking Includes is like a whitelist, and Excludes is like a blacklist.

  • When using Excludes, include all files that are not specified
  • When using Includes, include all specified files

If both are given, we can't decide which to prioritize.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got it. These are lists of filenames right, not regexes.
IMO, we can ignore this error by taking the excludes with a higher priority.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm unfamiliar with the standard, but is it general to prioritize the Excludes? I'm worried about letting users be confused.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there are no rules for that.
(or just a ref: https://gist.github.com/ento/cf5593d4fac3f72d3765b3fe2ed2a204)

But I think in our case, by missing/wrong-configuration, no deployment was triggered is better/safer than a deployment was triggered.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with that, okay, let's make Excludes top priority and will tell about the priority to users well.

@pipecd-bot
Copy link
Collaborator

COVERAGE

Code coverage for golang is 33.83%. This pull request increases coverage by 0.29%.

File Function Base Head Diff
pkg/config/image_watcher.go filterImageWatcherFiles -- 100.00% +100.00%
pkg/config/piped.go PipedImageWatcher.Validate -- 100.00% +100.00%
pkg/model/deployment.go DeploymentStatusesFromStrings -- 0.00% +0.00%
pkg/model/deployment.go DeploymentStatusStrings -- 0.00% +0.00%
pkg/app/piped/logpersister/stagelogpersister.go stageLogPersister.flushFromLastCheckpoint 50.00% 61.11% +11.11%
pkg/config/image_watcher.go LoadImageWatcher 0.00% 0.00% +0.00%
pkg/config/piped.go PipedSpec.Validate 56.25% 55.56% -0.69%
pkg/app/pipectl/cmd/application/add.go newAddCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/add.go add.run 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/application.go NewCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/sync.go newSyncCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/sync.go sync.run 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/sync.go makeStatuses 100.00% -- -100.00%
pkg/app/pipectl/cmd/application/sync.go availableStatuses 100.00% -- -100.00%
pkg/app/pipectl/cmd/deployment/deployment.go NewCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/deployment/waitstatus.go newWaitStatusCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/deployment/waitstatus.go waitStatus.run 0.00% -- +-0.00%
pkg/app/pipectl/cmd/deployment/waitstatus.go makeStatuses 100.00% -- -100.00%
pkg/app/pipectl/cmd/deployment/waitstatus.go availableStatuses 100.00% -- -100.00%

dir := filepath.Join(repoRoot, SharedConfigurationDirName)
files, err := ioutil.ReadDir(dir)
if err != nil {
return nil, false, fmt.Errorf("failed to read %s: %w", dir, err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think no .pipe directory is a normal case. In that case, is piped sending this error to the log system?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly, it should be handled as not found case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should compare with os.IsNotExist.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right

if _, ok := whiteList[f.Name()]; ok {
filtered = append(filtered, f)
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return to avoid else

Copy link
Member Author

@nakabonne nakabonne Dec 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expected it will be pointed out by you lol
It's tough to go out into it in English, but I'd say these processes are quite similar, so using if-else makes it readble. But it depends on the reader. What do you think?

Copy link
Member

@nghialv nghialv Dec 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😄
I always follow return-early and avoid-deep-nesting.
Because when we see a return we don't need to care about all of the later parts.
Using if-else we don't have to care about the else's inside but still have to check the part after that else.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha 😄
No objection with that philosophy!

@nakabonne
Copy link
Member Author

@nghialv Fixed them!

@pipecd-bot
Copy link
Collaborator

COVERAGE

Code coverage for golang is 33.81%. This pull request increases coverage by 0.27%.

File Function Base Head Diff
pkg/config/image_watcher.go filterImageWatcherFiles -- 100.00% +100.00%
pkg/config/piped.go PipedImageWatcher.Validate -- 100.00% +100.00%
pkg/model/deployment.go DeploymentStatusesFromStrings -- 0.00% +0.00%
pkg/model/deployment.go DeploymentStatusStrings -- 0.00% +0.00%
pkg/config/piped.go PipedSpec.Validate 56.25% 55.56% -0.69%
pkg/app/piped/logpersister/stagelogpersister.go stageLogPersister.flushFromLastCheckpoint 50.00% 61.11% +11.11%
pkg/config/image_watcher.go LoadImageWatcher 0.00% 0.00% +0.00%
pkg/app/pipectl/cmd/application/add.go newAddCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/add.go add.run 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/application.go NewCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/sync.go newSyncCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/sync.go sync.run 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/sync.go makeStatuses 100.00% -- -100.00%
pkg/app/pipectl/cmd/application/sync.go availableStatuses 100.00% -- -100.00%
pkg/app/pipectl/cmd/deployment/deployment.go NewCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/deployment/waitstatus.go newWaitStatusCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/deployment/waitstatus.go waitStatus.run 0.00% -- +-0.00%
pkg/app/pipectl/cmd/deployment/waitstatus.go makeStatuses 100.00% -- -100.00%
pkg/app/pipectl/cmd/deployment/waitstatus.go availableStatuses 100.00% -- -100.00%

@pipecd-bot
Copy link
Collaborator

COVERAGE

Code coverage for golang is 33.80%. This pull request increases coverage by 0.26%.

File Function Base Head Diff
pkg/config/image_watcher.go filterImageWatcherFiles -- 100.00% +100.00%
pkg/config/piped.go PipedImageWatcher.Validate -- 100.00% +100.00%
pkg/model/deployment.go DeploymentStatusesFromStrings -- 0.00% +0.00%
pkg/model/deployment.go DeploymentStatusStrings -- 0.00% +0.00%
pkg/app/piped/logpersister/stagelogpersister.go stageLogPersister.flushFromLastCheckpoint 50.00% 61.11% +11.11%
pkg/config/image_watcher.go LoadImageWatcher 0.00% 0.00% +0.00%
pkg/config/piped.go PipedSpec.Validate 56.25% 55.56% -0.69%
pkg/app/pipectl/cmd/application/add.go newAddCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/add.go add.run 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/application.go NewCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/sync.go newSyncCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/sync.go sync.run 0.00% -- +-0.00%
pkg/app/pipectl/cmd/application/sync.go makeStatuses 100.00% -- -100.00%
pkg/app/pipectl/cmd/application/sync.go availableStatuses 100.00% -- -100.00%
pkg/app/pipectl/cmd/deployment/deployment.go NewCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/deployment/waitstatus.go newWaitStatusCommand 0.00% -- +-0.00%
pkg/app/pipectl/cmd/deployment/waitstatus.go waitStatus.run 0.00% -- +-0.00%
pkg/app/pipectl/cmd/deployment/waitstatus.go makeStatuses 100.00% -- -100.00%
pkg/app/pipectl/cmd/deployment/waitstatus.go availableStatuses 100.00% -- -100.00%

@nakabonne
Copy link
Member Author

nakabonne commented Dec 15, 2020

@nghialv Fixed done.

@nghialv
Copy link
Member

nghialv commented Dec 15, 2020

/approve

@pipecd-bot
Copy link
Collaborator

APPROVE

This pull request is APPROVED by nghialv.

Approvers can cancel the approval by writing /approve cancel in a comment. Any additional commits also will change this pull request to be not-approved.

@pipecd-bot pipecd-bot merged commit 4d0cfb9 into master Dec 15, 2020
@pipecd-bot pipecd-bot deleted the image-watcher-config branch December 15, 2020 08:26
@pipecd-bot pipecd-bot mentioned this pull request Dec 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Load image watcher config
3 participants