New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mbed import treats URLs differently differently than git clone #706

Open
mtfurlan opened this Issue Jun 22, 2018 · 11 comments

Comments

Projects
None yet
5 participants
@mtfurlan
Copy link

mtfurlan commented Jun 22, 2018

I would like to import from a git repo on a computer.

mbed import --protocol git user@host:path/to/repo/from/home/dir

This fails, because it tries to clone from ssh://user@host/path/to/repo/from/home/dir
It then proceeds to try mercurial, and complains that mercurial isn't installed in spite of passing --protocol
If you do

git clone user@host:path/to/repo/from/home/dir

It effectively expands (no idea how it actually does it) to ssh://user@host/home/user/path/to/repo/from/home/dir
So the correct command is

mbed import user@host:/home/user/path/to/repo/from/home/dir

The moral of this story is I did mbed import --protocol git user@host:path/to/repo/from/home/dir and got the error that mercurial was not installed.

I'm not sure what a good solution to this is, because we can't assume /home/user, correct?

It was just fairly confusing.

Also how to get help on a subcommand is really unclear.
mbed help makes sense, so I tried mbed help import, mbed import help, mbed --help import, and then asssumed there was no help for subcommands.
I proceeded to start reading the source to figure out how the protocol was set, and I found what should be help for subcommands.
I started reading up on how the argparser library worked to try to suggest a fix, and saw that some tutorial used the invocation command subcommand --help.
Maybe mention that in the results of mbed help

@theotherjimmy

This comment has been minimized.

Copy link
Collaborator

theotherjimmy commented Jun 22, 2018

@mtfurlan The confusion comes from the fact that git is not a protocol. ssh and http and https are the protocols used. You don't specify the VCS to use when running mbed import

The moral of this story is I did mbed import --protocol git user@host:path/to/repo/from/home/dir and got the error that mercurial was not installed.

Mercurial is a dependency, so that makes sense. (the --protocol git is effectively a noop).

I'm not sure what a good solution to this is, because we can't assume /home/user, correct?

I have no idea what "we can't assume /home/user" is supposed to mean

... command subcommand --help.
Maybe mention that in the results of mbed help

Agreed. Let's use this issue to track that task.
[Mirrored to Jira]

@screamerbg

This comment has been minimized.

Copy link
Member

screamerbg commented Jun 22, 2018

@mtfurlan Somewhat repeating what @theotherjimmy said, Git's documentation outlines that:

The SSH Protocol

A common transport protocol for Git when self-hosting is over SSH. This is because SSH access to servers is already set up in most places — and if it isn’t, it’s easy to do. SSH is also an authenticated network protocol and, because it’s ubiquitous, it’s generally easy to set up and use.

To clone a Git repository over SSH, you can specify an ssh:// URL like this:

$ git clone ssh://[user@]server/project.git

Or you can use the shorter scp-like syntax for the SSH protocol:

$ git clone [user@]server:project.git

https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols

Regarding the host/user/home conversion, could you please attach a log and mask the actual IP/hostname?

Lastly the help command can be easily extended to support that. Thanks for the suggestion
[Mirrored to Jira]

@mtfurlan

This comment has been minimized.

Copy link
Author

mtfurlan commented Jun 22, 2018

It's true that git is not a protocol, I mean that mbed import treats URLs differently than git clone.

I took a URL I can pass to git clone, and mbed import gave unrelated errors. (Yes mercurial is a dependency, but that is completely unrelated to why it can't clone my git repo)

On --protocol, the help claims it exists, but yeah it's completely unused right now right now.
https://github.com/ARMmbed/mbed-cli/blob/master/mbed/mbed.py#L1877

And by "we can't assume /home/user" I meant that I don't know if we can take

mbed import user@host:path/to/repo/from/home/dir

and autoconvert it to

mbed import user@host:/home/user/path/to/repo/from/home/dir

because there is no leading / after user@host:

[Mirrored to Jira]

@mtfurlan

This comment has been minimized.

Copy link
Author

mtfurlan commented Jun 22, 2018

@screamerbg I'm not sure what log you want, here is the very verbose output of mbed import:

$ rm -rf marco/; mbed import -vv mark@10.4.4.198:advanced/b2v/ble5/mbed/marco
[mbed] Working path "/home/mark" (directory)
[mbed] Importing program "marco" from "ssh://mark@10.4.4.198/advanced/b2v/ble5/mbed/marco" at latest revision in the current branch
[mbed] Trying to guess source control management tool. Supported SCMs: bld, git, hg
[mbed] Exec "git clone ssh://mark@10.4.4.198/advanced/b2v/ble5/mbed/marco /home/mark/marco -v" in /home/mark
Cloning into '/home/mark/marco'...
Warning: Permanently added '10.4.4.198' (ECDSA) to the list of known hosts.
mark@10.4.4.198's password:
fatal: '/advanced/b2v/ble5/mbed/marco' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
[mbed] Exec "hg clone ssh://mark@10.4.4.198/advanced/b2v/ble5/mbed/marco /home/mark/marco -v" in /home/mark
[mbed] ERROR: Could not execute "hg".
[mbed] ERROR: Please verify that it's installed and accessible from your current path by executing "hg".
---

So yeah the issue is using the scp-like syntax with mbed.
Thinking a bit better, not converting the URL in formaturl from the scp-like syntax would be best.
[Mirrored to Jira]

@ciarmcom ciarmcom added the mirrored label Jun 22, 2018

@screamerbg

This comment has been minimized.

Copy link
Member

screamerbg commented Jun 22, 2018

@mtfurlan According to the log, mark@10.4.4.198:advanced/b2v/ble5/mbed/marco is converted to ssh://mark@10.4.4.198/advanced/b2v/ble5/mbed/marco, which is the correct behavior. E.g. can you try to clone with the ssh schema URL, e.g. git clone ssh://mark@10.4.4.198/advanced/b2v/ble5/mbed/marco?

Regarding the path handling the remote host host, if user mark has the home dir under /raid/mark/ then the actual remote folder would end up to be `/raid/mark/advanced/b2v/ble5/mbed/marco. Not sure if absolute paths should be used at all in this case. Could you try the following:

  • $ git clone ssh://mark@10.4.4.198/advanced/b2v/ble5/mbed/marco
  • $ git clone ssh://mark@10.4.4.198:advanced/b2v/ble5/mbed/marco
  • $ git clone ssh://mark@10.4.4.198:/advanced/b2v/ble5/mbed/marco

And let me know which one works?
[Mirrored to Jira]

@mtfurlan

This comment has been minimized.

Copy link
Author

mtfurlan commented Jun 22, 2018

  • $ git clone ssh://mark@10.4.4.198/advanced/b2v/ble5/mbed/marco
    • fatal: '/advanced/b2v/ble5/mbed/marco' does not appear to be a git repository
  • $ git clone ssh://mark@10.4.4.198:advanced/b2v/ble5/mbed/marco
    • ssh: Could not resolve hostname 10.4.4.198:advanced: Name or service not known
  • $ git clone ssh://mark@10.4.4.198:/advanced/b2v/ble5/mbed/marco
    • fatal: '/advanced/b2v/ble5/mbed/marco' does not appear to be a git repository
  • $ git clone mark@10.4.4.198:advanced/b2v/ble5/mbed/marco
    • Works

The repo is in /home/mark/advanced/b2v/ble5/mbed/marco

[Mirrored to Jira]

@screamerbg

This comment has been minimized.

Copy link
Member

screamerbg commented Jun 25, 2018

@mtfurlan Thanks for this. Could you also try:

  • git clone ssh://mark@10.4.4.198/home/mark/advanced/b2v/ble5/mbed/marco
  • git clone ssh://mark@10.4.4.198/~advanced/b2v/ble5/mbed/marco

I'm trying to understand how this deviates from the Git official documentation. There is no mention at https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols#_the_ssh_protocol regarding paths handling.
[Mirrored to Jira]

@mtfurlan

This comment has been minimized.

Copy link
Author

mtfurlan commented Jun 25, 2018

The first one works, the second one doesn't.

Yeah, the docs just say it supports "scp-like syntax", which is the home dir if not specified. I can't find the official scp documentation specifying this syntax (I didn't look much past the man page and a few google results), but there are a lot of examples of user@host:fileInHomeDir online.

What is the reason mbed converts URLS to ssh://? Git doesn't convert them.
Using mark@10.4.4.198:advanced/b2v/ble5/mbed/marco

$ git remote -v
origin  mark@10.4.4.198:advanced/b2v/ble5/mbed/marco (fetch)
origin  mark@10.4.4.198:advanced/b2v/ble5/mbed/marco (push)

[Mirrored to Jira]

@ARMmbed ARMmbed deleted a comment from ciarmcom Oct 2, 2018

@theotherjimmy

This comment has been minimized.

Copy link
Collaborator

theotherjimmy commented Oct 30, 2018

@mtfurlan @screamerbg Having read through this discussion, I'm left with the question: what action is left for this issue? Do you have an answer for this question?

@mtfurlan

This comment has been minimized.

Copy link
Author

mtfurlan commented Oct 30, 2018

I don't fully understand the syntax for how git treats URLs, so I don't want to make my own fix that makes different edge cases, that's how we ended up with the current implementation.

Next steps are to either

  • Document that mbed treats URLs differently than git
  • Learn how git handles URLs and copy that.

I did a cursory search for a description of how git handles URLs during the active discussion, but my google fu is weak, and I still don't have the time to dedicate to reading the source.

@theotherjimmy

This comment has been minimized.

Copy link
Collaborator

theotherjimmy commented Oct 30, 2018

@mtfurlan Thanks for that. I'll document how mbed treats URLs, and note that git treats them differently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment