Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

CLI: Download Detector Configuration as File #229

Merged
merged 6 commits into from
Sep 17, 2020

Conversation

VijayanB
Copy link
Member

Download configuration as files helps users to save it in source code control and maintains it.
This will also let users to share detector configuration across users.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@codecov
Copy link

codecov bot commented Sep 12, 2020

Codecov Report

Merging #229 into master will decrease coverage by 0.14%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #229      +/-   ##
============================================
- Coverage     72.40%   72.25%   -0.15%     
+ Complexity     1289     1278      -11     
============================================
  Files           139      139              
  Lines          6073     6045      -28     
  Branches        469      469              
============================================
- Hits           4397     4368      -29     
- Misses         1464     1465       +1     
  Partials        212      212              
Flag Coverage Δ Complexity Δ
#plugin 71.17% <ø> (-0.17%) 1278.00 <ø> (-11.00)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ Complexity Δ
...opendistroforelasticsearch/ad/ml/ModelManager.java 90.93% <0.00%> (-1.13%) 100.00% <0.00%> (-12.00%)
cli/internal/controller/es/es.go 100.00% <0.00%> (ø) 0.00% <0.00%> (ø%)
...sticsearch/ad/indices/AnomalyDetectionIndices.java 62.31% <0.00%> (+0.72%) 24.00% <0.00%> (+1.00%)
cli/internal/handler/ad/ad.go 90.09% <0.00%> (ø) 0.00% <0.00%> (ø%)
cli/internal/gateway/es/es.go 86.66% <0.00%> (ø) 0.00% <0.00%> (ø%)
cli/internal/mapper/ad/ad.go 86.42% <0.00%> (ø) 0.00% <0.00%> (ø%)
cli/internal/controller/ad/ad.go 77.38% <0.00%> (ø) 0.00% <0.00%> (ø%)
cli/internal/gateway/ad/ad.go 65.38% <0.00%> (ø) 0.00% <0.00%> (ø%)

Go provides an abstraction to support writing on file
and stdout as same interface, refactor cat to accept
func as input. Later, download will pass writeInfile as parameter
@VijayanB VijayanB changed the title Download Detector Configuration as File CLI: Download Detector Configuration as File Sep 14, 2020
cli/cmd/cat.go Outdated
Comment on lines 51 to 54
if err != nil {
fmt.Println(err)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack

Printing error on console is added everywhere.
Create a function to check error and print.
Download is similar to cat, excepts it download the configuration and
save it as file with detector name as file name on current folder
if err != nil {
return err
}
filePath := filepath.Join(cwd, d.Name)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add file extension?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added .json

return err
}
filePath := filepath.Join(cwd, d.Name)
f, err := os.Create(filePath)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the filePath exists, will os.Create override the existing file?

Copy link
Contributor

@ylwu-amzn ylwu-amzn Sep 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From this doc, https://golang.org/pkg/os/#Create

If the file already exists, it is truncated

How about check file exists or not before create ? If file exists, we should not truncate it directly in case user execute command by mistake.

Copy link
Member Author

@VijayanB VijayanB Sep 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I introduced a flag -i similar to cp in linux commands, which prompt to stdout whether to overwrite the file or not.
Screen Shot 2020-09-16 at 3 40 49 PM

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The flag -i is optional? What if user don't add -i?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is interactive. By default download will overwrite but if user passes -i, it will prompt for user confirmation. Do you think prompt should be default?

Copy link
Contributor

@ylwu-amzn ylwu-amzn Sep 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think prompt should be default for this case, not good user experience/safe to override user's existing file without any prompt.

//file will be created inside current working directory,
//with detector name as file name
func WriteInFile(d *entity.DetectorOutput) error {
cwd, err := os.Getwd()
Copy link
Contributor

@ylwu-amzn ylwu-amzn Sep 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to output detector into a different directory other than current directory? Can user specify the absolute file path? For example, user want to download detector and output it into "/tmp/my_detectors.json".

If user query detector with name pattern and return multiple detectors, we will write detectors into separate files. Seems not very easy for user to share if they have hundreds of such small detector configuration files. By specifying an absolute file path and output all detectors' configuration into one file will be easier for user to share, bulk recreate etc.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes definitely possible. I am planning to include this feature as next PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added allow user to provide output folder as new commit.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, thanks for the change

Similar to cp from linux, add interactive to allow user to decide
whether to overwrite file or not
Added flag to allow user to specify target location where
detectors will be downloaded.
Download detector will always prompt if there is file
exists in output directory.
Copy link
Contributor

@ylwu-amzn ylwu-amzn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the change.

@VijayanB VijayanB merged commit 7d780a8 into opendistro-for-elasticsearch:master Sep 17, 2020
@VijayanB VijayanB deleted the download branch September 17, 2020 17:37
@ohltyler ohltyler added the enhancement New feature or request label Oct 19, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants