Skip to content

Allow stderr for git-annex subprocess with records generator #7329

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 24, 2023

Conversation

bpoldrack
Copy link
Member

@bpoldrack bpoldrack commented Mar 15, 2023

The previous assertion that there's no stderr output to be expected is
wrong. This function is calling git-annex commands, which in turn may
involve arbitrary special remotes. DataLad can not know whether there is
anything to expect on stderr and if such would indicate an error. We
should be able to rely on git-annex returning non-zero.

Closes #7328

@bpoldrack bpoldrack marked this pull request as draft March 15, 2023 14:42
@bpoldrack
Copy link
Member Author

FTR: MacOS failures during setup, Ubuntu failures are #7320

@bpoldrack
Copy link
Member Author

@christian-monch : Could you have a look whether that makes sense to you?

Are there any side-effects you can think of, when switching those protocols? Only aim is not failing just b/c there's some output on stderr.

@christian-monch
Copy link
Contributor

@christian-monch : Could you have a look whether that makes sense to you?

Are there any side-effects you can think of, when switching those protocols? Only aim is not failing just b/c there's some output on stderr.

Functionally it should be identical, except from the fact that stderr-output does not lead to an exception.

Having said that, the only reason that the GeneratorAnnexJsonNoStderrProtocol-class was introduced was that the original code raised an exception when stderr-output was received.

@bpoldrack
Copy link
Member Author

Thanks for confirming!

the original code raised an exception when stderr-output was received

Yes, that may well be. But I think it's the wrong thing to do. We are invoking arbitrary annex commands. We cannot possibly know whether any stderr output is expected, especially when annex itself is going to call arbitrary special remote implementations.

The previous assertion that there's no stderr output to be expected is
wrong. This function is calling git-annex commands, which in turn may
involve arbitrary special remotes. DataLad can not know whether there is
anything to expect on stderr and if such would indicate an error. We
should be able to rely on git-annex returning non-zero.

Closes datalad#7328
@bpoldrack bpoldrack changed the title TMP: Test protocol switch Allow stderr for git-annex subprocess with records generator Mar 16, 2023
@bpoldrack bpoldrack added the semver-patch Increment the patch version when merged label Mar 16, 2023
@bpoldrack bpoldrack marked this pull request as ready for review March 16, 2023 10:32
@bpoldrack bpoldrack merged commit 7557591 into datalad:maint Mar 24, 2023
@yarikoptic-gitmate
Copy link
Collaborator

PR released in 0.18.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver-patch Increment the patch version when merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

_call_annex_records_items_ incompatible with logging
3 participants