Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

commit,completion: Add stages completion for dvc commit #5963

Merged
merged 1 commit into from
May 5, 2021

Conversation

ciaochiaociao
Copy link
Contributor

Fixes #5960

Thank you for the contribution - we'll try to review it as soon as possible. 🙏

Copy link
Member

@efiop efiop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! 🙏

@@ -71,5 +71,5 @@ def add_parser(subparsers, parent_parser):
nargs="*",
help="stages or .dvc files to commit. Optional. "
"(Finds all DVC files in the workspace by default.)",
).complete = completion.DVC_FILE
).complete = completion.DVCFILES_AND_STAGE
Copy link
Member

@skshetry skshetry May 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ciaochiaociao, do you use autocompletion of the stage for the repro?

This might end up being slower for you. Do you perceive any lag?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I do. Slower after I enable completion of stages for dvc commit? dvc repro already had stages completion enabled, while dvc commit did not. And yes, there are more lags indeed for dvc commit currently. Now I have like 20 stages, and it takes me like 1~2 seconds to show the completion suggestions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1-2 seconds seem to be quite bad for just 20 stages. If it's possible, could you please post the output of hyperfine "dvc stage list" --warmup 3 or just run the time dvc stage list inside your repo. I get ~500ms for a similar-sized file.

Usually, users use the filename as a target in the commit. That's why I was asking. Thanks for the PR. 🙂

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$ time dvc stage list
parse@train                  Outputs dvc_pipeline/parse_train
parse@dev                    Outputs dvc_pipeline/parse_dev
parse@test                   Outputs dvc_pipeline/parse_test
locate_candidates@train      Outputs dvc_pipeline/candidates_train
locate_candidates@dev        Outputs dvc_pipeline/candidates_dev
locate_candidates@test       Outputs dvc_pipeline/candidates_test
parse_cand_log@train         Outputs dvc_pipeline/candidates_train_logs; Reports metrics/candidates_train.ya…
parse_cand_log@dev           Outputs dvc_pipeline/candidates_dev_logs; Reports metrics/candidates_dev.yaml
parse_cand_log@test          Outputs dvc_pipeline/candidates_test_logs; Reports metrics/candidates_test.yaml
IR@train                     Outputs dvc_pipeline/IR_train
IR@dev                       Outputs dvc_pipeline/IR_dev
IR@test                      Outputs dvc_pipeline/IR_test
build_serpapi_queries@train  Outputs dvc_pipeline/serpapi_queries_train
build_serpapi_queries@dev    Outputs dvc_pipeline/serpapi_queries_dev
build_serpapi_queries@test   Outputs dvc_pipeline/serpapi_queries_test
merge_serpapi_queries        Outputs dvc_pipeline/serpapi_queries/queries.txt
run_serpapi                  Outputs dvc_pipeline/run_serpapi/cache.json, dvc_pipeline/run_serpapi/debug.log…
ner_lucene@train             Outputs dvc_pipeline/ner_lucene_train
ner_lucene@dev               Outputs dvc_pipeline/ner_lucene_dev
ner_lucene@test              Outputs dvc_pipeline/ner_lucene_test
ner_serpapi@train            Outputs dvc_pipeline/ner_serpapi_train
ner_serpapi@dev              Outputs dvc_pipeline/ner_serpapi_dev
ner_serpapi@test             Outputs dvc_pipeline/ner_serpapi_test
build_corpus                 Outputs dvc_pipeline/final
parse_build_corpus_log       Reports metrics/build_corpus.yaml
eval_corpus                  Outputs dvc_pipeline/eval_corpus_run.ipynb
train_recovery               Outputs dvc_model, dvc_tb
parse_train_logs             Reports metrics/train_recovery.yaml
print_recovered@train        Outputs dvc_pipeline/print_recovered_train.ipynb
print_recovered@dev          Outputs dvc_pipeline/print_recovered_dev.ipynb
print_recovered@test         Outputs dvc_pipeline/print_recovered_test.ipynb
analyze_recovery@train       Outputs dvc_pipeline/analyze_recovery_train.ipynb; Reports metrics/analyze_reco…
analyze_recovery@dev         Outputs dvc_pipeline/analyze_recovery_dev.ipynb; Reports metrics/analyze_recove…
analyze_recovery@test        Outputs dvc_pipeline/analyze_recovery_test.ipynb; Reports metrics/analyze_recov…
analyze_theoretically@train  Reports metrics/analyze_theoretically_train.yaml
analyze_theoretically@dev    Reports metrics/analyze_theoretically_dev.yaml
analyze_theoretically@test   Reports metrics/analyze_theoretically_test.yaml
dvc stage list  0.42s user 0.03s system 72% cpu 0.627 total

More than 20 stages, about 40 actually.
By the way, I am using oh-my-zsh with spaceship theme. This theme does slow a bit compared to a pure zsh. Just FYI.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dvc stage list 0.42s user 0.03s system 72% cpu 0.627 total

Does not look that bad, not much we can do to speed it up right now sadly.

@efiop efiop merged commit 3670bf4 into iterative:master May 5, 2021
@efiop efiop added the enhancement Enhances DVC label May 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhances DVC
Projects
None yet
Development

Successfully merging this pull request may close these issues.

commit: doesn't show completion for stages in zsh
3 participants