Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

p4: fix "Not a valid object name HEAD0" when unshelving #183

Closed
wants to merge 1 commit into from

Conversation

@mdymike
Copy link

mdymike commented May 3, 2019

git p4 unshelve was failing with "fatal: Not a valid object name HEAD0" and "Command failed: git cat-file commit HEAD^0" on certain systems e.g. git version 2.21.0.windows.1 + python 2.7.16

It seems that certain python pOpen implementations drop the ^ character when invoked using a string instead of an array as first argument, which is what is done by extractLogMessageFromGitCommit.

Solution is to use the array format of passing the command to fOpen, which is preferred (see https://docs.python.org/2/library/subprocess.html) and is used in other parts of this code anyway.

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented May 3, 2019

Welcome to GitGitGadget

Hi @mdymike, and welcome to GitGitGadget, the GitHub App to send patch series to the Git mailing list from GitHub Pull Requests.

Please make sure that this Pull Request has a good description, as it will be used as cover letter.

Also, it is a good idea to review the commit messages one last time, as the Git project expects them in a quite specific form:

  • the lines should not exceed 76 columns,
  • the first line should be like a header and typically start with a prefix like "tests:" or "commit:", and
  • the commit messages' body should be describing the "why?" of the change.
  • Finally, the commit messages should end in a Signed-off-by: line matching the commits' author.

It is in general a good idea to await the automated test ("Checks") in this Pull Request before contributing the patches, e.g. to avoid trivial issues such as unportable code.

Contributing the patches

Before you can contribute the patches, your GitHub username needs to be added to the list of permitted users. Any already-permitted user can do that, by adding a PR comment of the form /allow <username>.

Once on the list of permitted usernames, you can contribute the patches to the Git mailing list by adding a PR comment /submit.

After you submit, GitGitGadget will respond with another comment that contains the link to the cover letter mail in the Git mailing list archive. Please make sure to monitor the discussion in that thread and to address comments and suggestions.

If you do not want to subscribe to the Git mailing list just to be able to respond to a mail, you can download the mbox ("raw") file corresponding to the mail you want to reply to from the Git mailing list. If you use GMail, you can upload that raw mbox file via:

curl -g --user "<EMailAddress>:<Password>" --url "imaps://imap.gmail.com/INBOX" -T /path/to/raw.txt
@dscho

This comment has been minimized.

Copy link
Member

dscho commented May 6, 2019

/allow mdymike

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented May 6, 2019

User mdymike is now allowed to use GitGitGadget.

@dscho

This comment has been minimized.

Copy link
Member

dscho commented May 6, 2019

@mdymike please consider using your real email address (as the Git mailing list will most likely reject the patch if it has a fake email address in the author/Signed-off-by info).

Also, please wrap the commit message at 74 columns/line or less.

Finally, I think that you might want to add a bit more information about that ^ thing: in Windows' cmd, the caret is the escape character (not the backslash, for obvious reasons: it is the directory separator). So when passing a command line, the ^ will get interpreted as escape character under certain circumstances (such as using the official Python for Windows).

Is my take on this correct?

@mdymike

This comment has been minimized.

Copy link
Author

mdymike commented May 9, 2019

Thanks @dscho! Implemented your suggestions, yes you were right about ^ on Windows

@mdymike

This comment has been minimized.

Copy link
Author

mdymike commented May 9, 2019

Note there are a couple of other suspicious uses of ^ however I don't have access to a non-prod perforce to test these code paths:

line#1459, part of P4RollBack
system("git update-ref %s "%s^"" % (ref, ref))

line#1830, part of applyCommit
diff = read_pipe_lines("git diff-tree -r %s "%s^" "%s"" % (self.diffOpts, id, id))

line#3303, part of getCommitByP4Change, called by importNewBranch
earliestCommit = "^%s" % next
(which is used read_pipe call above)

If those lines have code coverage in the automatic checks I would be happy to propose the fixes, just let me know.

@mdymike

This comment has been minimized.

Copy link
Author

mdymike commented May 9, 2019

/submit

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented May 9, 2019

Error: Could not determine full name of mdymike

@dscho

This comment has been minimized.

Copy link
Member

dscho commented May 9, 2019

Error: Could not determine full name of mdymike

:-(

Sorry about that. GitGitGadget looks at your GitHub profile to come up with your full name so that the mails can be sent as "Mike Mueller via GitGitGadget". And since your profile does not have that information, GitGitGadget gives up.

I'm currently quite busy preparing for the upcoming v2.22.0-rc0, so I am unable to change GitGitGadget right now. Would you mind changing your profile and then issuing another /submit?

@mdymike

This comment has been minimized.

Copy link
Author

mdymike commented May 10, 2019

/submit

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented May 10, 2019

@@ -737,7 +737,7 @@ def extractLogMessageFromGitCommit(commit):

This comment has been minimized.

Copy link
@gitgitgadget

gitgitgadget bot May 15, 2019

On the Git mailing list, Junio C Hamano wrote (reply to this):

"Mike Mueller via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Mike Mueller <mike.mueller@moodys.com>
>
> git p4 unshelve was failing with these errors on Windows:
>
> fatal: Not a valid object name HEAD0
> Command failed: git cat-file commit HEAD^0
>
> (git version 2.21.0.windows.1, python 2.7.16)
>
> The pOpen call used by git-p4 to invoke the git command can take either a
> string or an array as a first argument.  The array form is preferred
> however the extractLogMessageFromGitCommit method was using the string
> form, which makes the caller responsible for escaping the command text
> appropriately (see https://docs.python.org/2/library/subprocess.html)

Rewrite the sentence that begin with "The array form is
preferred...", as it is somewhat unreadable.  "X is preferred
because Y; however Z was using the other one" would be
understandable.

> Somewhat ironically, the carat character is the escape character in

s/carat/caret/ everywhere.

> Windows and so should be escaped (HEAD^^0).  Without the extra carat, the
> OS was passing an escaped 0 to the git command instead, and the git
> command was rejecting the invalid object name "HEAD0"
>
> The behaviour can be confirmed by typing ECHO HEAD^0 at the command-
> prompt, which emits HEAD0.
>
> The solution is simply to use the array format of passing the command to
> fOpen, which is preferred and used in other parts of this code anyway.
>
> Signed-off-by: Mike Mueller <mike.mueller@moodys.com>
> ---
>  git-p4.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/git-p4.py b/git-p4.py
> index 5b79920f46..0b5bfcbc5e 100755
> --- a/git-p4.py
> +++ b/git-p4.py
> @@ -737,7 +737,7 @@ def extractLogMessageFromGitCommit(commit):
>  
>      ## fixme: title is first line of commit, not 1st paragraph.
>      foundTitle = False
> -    for log in read_pipe_lines("git cat-file commit %s" % commit):
> +    for log in read_pipe_lines(["git", "cat-file", "commit", commit]):
>         if not foundTitle:
>             if len(log) == 1:
>                 foundTitle = True
git p4 unshelve was failing with these errors:

fatal: Not a valid object name HEAD0
Command failed: git cat-file commit HEAD^0

(git version 2.21.0.windows.1, python 2.7.16)

The pOpen call used by git-p4 to invoke the git command can take either a
string or an array as a first argument. The array form is preferred
because platform-specific escaping of special characters will be
handled automatically.(https://docs.python.org/2/library/subprocess.html)
The extractLogMessageFromGitCommit method was, however, using the string
form and so the caret (^) character in the HEAD^0 argument was not being
escaped on Windows.  The caret happens to be the escape character, which
is why the git command was receiving HEAD0.

The behaviour can be confirmed by typing ECHO HEAD^0 at the command-
prompt, which emits HEAD0.

The solution is simply to use the array format of passing the command to
fOpen, which is recommended and used in other parts of this code anyway.

Signed-off-by: Mike Mueller <mike.mueller@moodys.com>
@mdymike

This comment has been minimized.

Copy link
Author

mdymike commented May 15, 2019

@dscho do you think this looks OK and do I need to /submit again?

@dscho

This comment has been minimized.

Copy link
Member

dscho commented May 17, 2019

do you think this looks OK and do I need to /submit again?

Looks okay to me (but it's up to @gitster to judge), and yes, you need to /submit again, as this will generate a v2 of the patch series (typically more than one adjustment is necessary, so it is common to force-push multiple times before submitting the next iteration).

@dscho

This comment has been minimized.

Copy link
Member

dscho commented May 18, 2019

it's up to @gitster to judge

FWIW the Git maintainer mentioned a couple of times that they strongly prefer to review patches on the Git mailing list, so there is no reason to wait with /submitting.

@mdymike

This comment has been minimized.

Copy link
Author

mdymike commented May 28, 2019

it's up to @gitster to judge

FWIW the Git maintainer mentioned a couple of times that they strongly prefer to review patches on the Git mailing list, so there is no reason to wait with /submitting.

Cool thanks - been offline in the mountains, will submit now - thanks!

@mdymike

This comment has been minimized.

Copy link
Author

mdymike commented May 28, 2019

/submit

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented May 28, 2019

@@ -737,7 +737,7 @@ def extractLogMessageFromGitCommit(commit):

This comment has been minimized.

Copy link
@gitgitgadget

gitgitgadget bot May 28, 2019

On the Git mailing list, Junio C Hamano wrote (reply to this):

"Mike Mueller via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Mike Mueller <mike.mueller@moodys.com>
>
> git p4 unshelve was failing with these errors:
>
> fatal: Not a valid object name HEAD0
> Command failed: git cat-file commit HEAD^0
>
> (git version 2.21.0.windows.1, python 2.7.16)
>
> The pOpen call used by git-p4 to invoke the git command can take either a
> string or an array as a first argument. The array form is preferred
> because platform-specific escaping of special characters will be
> handled automatically.(https://docs.python.org/2/library/subprocess.html)
> The extractLogMessageFromGitCommit method was, however, using the string
> form and so the caret (^) character in the HEAD^0 argument was not being
> escaped on Windows.  The caret happens to be the escape character, which
> is why the git command was receiving HEAD0.

In the output from

    git grep 'read_pipe_lines("'

together with a few hits to harmless constant command line, we find
this line

    diff = read_pipe_lines("git diff-tree -r %s \"%s^\" \"%s\"" % (self.diffOpts, id, id))

Would the caret we see there cause a similar problem?  It would end
up running something like

   $ git diff-tree -r -M "HEAD^" "HEAD"

This comment has been minimized.

Copy link
@gitgitgadget

gitgitgadget bot May 29, 2019

On the Git mailing list, Johannes Schindelin wrote (reply to this):

Hi Junio,

On Tue, 28 May 2019, Junio C Hamano wrote:

> "Mike Mueller via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > From: Mike Mueller <mike.mueller@moodys.com>
> >
> > git p4 unshelve was failing with these errors:
> >
> > fatal: Not a valid object name HEAD0
> > Command failed: git cat-file commit HEAD^0
> >
> > (git version 2.21.0.windows.1, python 2.7.16)
> >
> > The pOpen call used by git-p4 to invoke the git command can take eithe=
r a
> > string or an array as a first argument. The array form is preferred
> > because platform-specific escaping of special characters will be
> > handled automatically.(https://docs.python.org/2/library/subprocess.ht=
ml)
> > The extractLogMessageFromGitCommit method was, however, using the stri=
ng
> > form and so the caret (^) character in the HEAD^0 argument was not bei=
ng
> > escaped on Windows.  The caret happens to be the escape character, whi=
ch
> > is why the git command was receiving HEAD0.
>
> In the output from
>
>     git grep 'read_pipe_lines("'
>
> together with a few hits to harmless constant command line, we find
> this line
>
>     diff =3D read_pipe_lines("git diff-tree -r %s \"%s^\" \"%s\"" % (sel=
f.diffOpts, id, id))
>
> Would the caret we see there cause a similar problem?  It would end
> up running something like
>
>    $ git diff-tree -r -M "HEAD^" "HEAD"

I think you're right!

In addition, I wonder whether we would want to replace the `^` by a `~`,
which would have the same effect, but does not need quoting in Bash nor
CMD.

Ciao,
Dscho
@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented May 28, 2019

This branch is now known as mm/p4-unshelve-windows-fix.

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented May 28, 2019

This patch series was integrated into pu via git@b4f728e.

@gitgitgadget gitgitgadget bot added the pu label May 28, 2019
@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented May 29, 2019

This patch series was integrated into pu via git@c47cb04.

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented May 29, 2019

This patch series was integrated into pu via git@fe98c6f.

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented May 30, 2019

This patch series was integrated into pu via git@f61ed81.

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented May 30, 2019

This patch series was integrated into pu via git@ed433b5.

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented Jun 3, 2019

This patch series was integrated into pu via git@f016246.

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented Jun 3, 2019

This patch series was integrated into next via git@d45167d.

@gitgitgadget gitgitgadget bot added the next label Jun 3, 2019
@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented Jun 6, 2019

This patch series was integrated into pu via git@1a7fcc0.

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented Jun 7, 2019

This patch series was integrated into pu via git@3e299f3.

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented Jun 10, 2019

This patch series was integrated into pu via git@0727d70.

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented Jun 12, 2019

This patch series was integrated into pu via git@5142854.

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented Jun 13, 2019

This patch series was integrated into pu via git@780a40b.

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented Jun 14, 2019

This patch series was integrated into pu via git@ca2c63d.

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented Jun 18, 2019

This patch series was integrated into pu via git@add59c4.

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented Jun 18, 2019

This patch series was integrated into next via git@add59c4.

@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented Jun 18, 2019

This patch series was integrated into master via git@add59c4.

@gitgitgadget gitgitgadget bot added the master label Jun 18, 2019
@gitgitgadget gitgitgadget bot closed this Jun 18, 2019
@gitgitgadget

This comment has been minimized.

Copy link

gitgitgadget bot commented Jun 18, 2019

Closed via add59c4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.