Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Capture hints from Git #4338

Merged
merged 4 commits into from Mar 26, 2020
Merged

ENH: Capture hints from Git #4338

merged 4 commits into from Mar 26, 2020

Conversation

adswa
Copy link
Member

@adswa adswa commented Mar 24, 2020

Sometimes, Git has "hints" why pushing fails.
For example, if I attempt a push without integrating remote changes, it says

error: failed to push some refs to 'git@github.com:datalad-handbook/DataLad-101.git'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Integrate the remote changes (e.g.
hint: 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

datalad push swallowed these hints. In this PR I'm attempting to catch them and report all hints at the end of the result rendering.

A few examples:

datalad push --to blubb43
publish(error): . (dataset) [refs/heads/git-annex->blubb43:refs/heads/git-annex [rejected] (fetch first)]
Potential hints to solve encountered errors: 
1: Updates were rejected because the remote contains work that you do not have locally. This is usually caused by another repository pushing to the same ref. You may want to first integrate the remote changes (e.g., 'git pull ...') before pushing again. See the 'Note about fast-forwards' in 'git push --help' for details.

A situation with lots of the same hints

datalad push --to public                                                                     1 ↵
publish(error): . (dataset) [refs/heads/master->public:refs/heads/master [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_back_and_forth_in_time->public:refs/heads/sct_back_and_forth_in_time [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_computational_reproducibility->public:refs/heads/sct_computational_reproducibility [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_create_a_dataset->public:refs/heads/sct_create_a_dataset [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_datalad_rerun->public:refs/heads/sct_datalad_rerun [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_hide_content->public:refs/heads/sct_hide_content [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_input_and_output->public:refs/heads/sct_input_and_output [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_install_datasets->public:refs/heads/sct_install_datasets [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_keeping_track->public:refs/heads/sct_keeping_track [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_looking_without_touching->public:refs/heads/sct_looking_without_touching [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_modify_content->public:refs/heads/sct_modify_content [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_more_on_DYI_configurations->public:refs/heads/sct_more_on_DYI_configurations [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_more_on_dataset_nesting->public:refs/heads/sct_more_on_dataset_nesting [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_networking->public:refs/heads/sct_networking [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_populate_a_dataset->public:refs/heads/sct_populate_a_dataset [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_retrace_and_reenact->public:refs/heads/sct_retrace_and_reenact [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_stay_up_to_date->public:refs/heads/sct_stay_up_to_date [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_where_is_waldo->public:refs/heads/sct_where_is_waldo [rejected] (non-fast-forward)]
publish(error): . (dataset) [refs/heads/sct_yoda_project->public:refs/heads/sct_yoda_project [rejected] (non-fast-forward)]
Potential hints to solve encountered errors: 
1: Updates were rejected because the tip of your current branch is behind its remote counterpart. Integrate the remote changes (e.g. 'git pull ...') before pushing again. See the 'Note about fast-forwards' in 'git push --help' for details.
2: Updates were rejected because a pushed branch tip is behind its remote counterpart. Check out this branch and integrate the remote changes (e.g. 'git pull ...') before pushing again. See the 'Note about fast-forwards' in 'git push --help' for details.

And with no hints:

datalad push --to roommate                                                                   1 ↵
publish(ok): . (dataset) [refs/heads/git-annex->roommate:refs/heads/git-annex f158575..2b80295]      
publish(error): . (dataset) [refs/heads/master->roommate:refs/heads/master [remote rejected] (branch is currently checked out)]

side effects:

We found out that the custom result summary renderer of get is never executed, because it would previously only be called if result_renderer was set to tailored. With b8b282d, it should be called now.

@adswa
Copy link
Member Author

adswa commented Mar 24, 2020

Oh, I broke things... but with the merged WTF warning it seems...

ERROR: datalad.plugin.tests.test_plugins.test_wtf

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/nose/case.py", line 198, in runTest

    self.test(*self.arg)

  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/datalad/tests/utils.py", line 518, in newfunc

    return t(*(arg + (d,)), **kw)

  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/datalad/plugin/tests/test_plugins.py", line 75, in test_wtf

    wtf(dataset=path)

  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/datalad/interface/utils.py", line 492, in eval_func

    return return_func(generator_func)(*args, **kwargs)

  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/datalad/interface/utils.py", line 480, in return_func

    results = list(results)

  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/datalad/interface/utils.py", line 467, in generator_func

    msg="Command did not complete successfully")

datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully [{'action': 'wtf', 'path': '/tmp/datalad_temp_tree_test_wtfnp0f08er/ "\';a&b&cΔЙקم๗あ `| ', 'status': 'impossible', 'message': ('No dataset found at %s. Reporting on the dataset is not attempted.', '/tmp/datalad_temp_tree_test_wtfnp0f08er/ "\';a&b&cΔЙקم๗あ `| ')}]


Copy link
Collaborator

@kyleam kyleam left a comment

quick minor comments from just scanning the diff but not thinking about the overall change

datalad/interface/utils.py Outdated Show resolved Hide resolved
@@ -2504,6 +2506,9 @@ def _fetch_push_helper(
pi = info_cls._from_line(line)
if add_remote:
pi['remote'] = remote
# There were errors, but Git provided hints
if 'error' in pi['operations']:
pi['hints'] = hints if hints is not '' else None
Copy link
Collaborator

@kyleam kyleam Mar 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is should be avoided because you're comparing a string literal. (IIRC Python actually started warning about this in a very recent version, perhaps 3.8.)

You could use ==, but in this case I think hints or None would suffice.

Copy link
Member Author

@adswa adswa Mar 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. To be honest, I don't know how/why this (hints or None) works. What can I google to find out about this?

Copy link
Collaborator

@kyleam kyleam Mar 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I first checked pydoc or. Ignoring the BNF notation and skipping down to the "x or y" paragraph, I think that description is pretty good:

The expression "x or y" first evaluates *x*; if *x* is true, its value
is returned; otherwise, *y* is evaluated and the resulting value is
returned.

So, x if x else y but slightly shorter (and more easily parsed, depending on the eyes).

Searching online for "python boolean operations", I had to scroll past a few tutorials, but not too far down saw a link to this part of the python docs, which says the same thing as the pydoc or page, perhaps more clearly.

@codecov
Copy link

codecov bot commented Mar 24, 2020

Codecov Report

Merging #4338 into master will increase coverage by 0.05%.
The diff coverage is 50.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4338      +/-   ##
==========================================
+ Coverage   88.85%   88.91%   +0.05%     
==========================================
  Files         285      285              
  Lines       37491    37553      +62     
==========================================
+ Hits        33313    33390      +77     
+ Misses       4178     4163      -15     
Impacted Files Coverage Δ
datalad/core/distributed/push.py 86.19% <11.11%> (-3.37%) ⬇️
datalad/interface/utils.py 93.67% <100.00%> (ø)
datalad/support/gitrepo.py 90.78% <100.00%> (+0.01%) ⬆️
datalad/support/exceptions.py 83.76% <0.00%> (ø)
datalad/support/tests/test_fileinfo.py 100.00% <0.00%> (ø)
datalad/cmd.py 89.92% <0.00%> (+0.01%) ⬆️
datalad/support/tests/test_annexrepo.py 95.32% <0.00%> (+0.04%) ⬆️
datalad/plugin/wtf.py 82.27% <0.00%> (+0.07%) ⬆️
datalad/support/annexrepo.py 86.31% <0.00%> (+0.26%) ⬆️
... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7bb9866...3f16fda. Read the comment docs.

mih
mih approved these changes Mar 26, 2020
Copy link
Member

@mih mih left a comment

Cool, thanks! I believe the crippled FS test failure is unrelated.

@mih mih merged commit 4a3035a into datalad:master Mar 26, 2020
10 of 12 checks passed
@adswa adswa deleted the enh-hints branch Dec 18, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants