Invalid repository name (sequenceiq\hadoop-docker), only [a-z0-9-_.] are allowed

[medined]

winpty seems great but I have run to a problem when using it with Docker. It seems like winpty is not passing the image names with slashes to Docker. I tried different combinations of quotes and slashes.

The docker command runs correctly on my Ubuntu laptop.

Any suggestions?

$ winpty docker run -it sequenceiq/hadoop-docker:2.7.0 /etc/bootstrap.sh -bash
Unable to find image 'sequenceiq\hadoop-docker:2.7.0' locally
FATA[0000] Invalid repository name (sequenceiq\hadoop-docker), only [a-z0-9-_.] are allowed

[rprichard]

Somewhat speculative:

If I understand correctly, MSYS/MSYS2 does a special argument translation when it invokes a non-MSYS/MSYS2 program. The translation does conversions like:

• /c/foo to C:/foo
• /foo to C:/msys32/foo

Cygwin doesn't do this translation, but instead provides a tool, cygpath, that users can use.

I think the docker.exe program is a plain Windows console program (i.e. not MSYS, MSYS2, Cygwin, etc).

This repository builds a console.exe rather than a winpty.exe, and console.exe only does a Unix-to-Windows path translation on the first argument, following the Cygwin convention.

When winpty is built for MSYS/MSYS2, it should probably follow the MSYS/MSYS2 convention, but then it will do the wrong thing in your case. I suspect that Docker's version of winpty did add the translation, but I haven't dug into it yet. winpty is packaged for MSYS2; maybe that packaging adds the translation step. The easiest way to implement the translation would be to call convertPosixPathToWin on every argument instead of just the first, but that wouldn't match the MSYS/MSYS2 convention w.r.t. the slash direction.

[Aside: it's a little curious that Docker is translating the VM's terminal output to console API calls, then using winpty to reconstruct terminal output from the console screen buffer. In principle, Docker could feed the VM terminal output directly to mintty.]

[medined]

I just tried to use the backward slash with the idea that it would be translated into the forward slash. And it changed the result, sort of. For the record, here is the result:

$ winpty docker run -it sequenceiq\hadoop-docker:2.7.0 /etc/bootstrap.sh -bash
Unable to find image 'sequenceiqhadoop-docker:2.7.0' locally
Pulling repository docker.io/library/sequenceiqhadoop-docker
FATA[0000] Error: image library/sequenceiqhadoop-docker:2.7.0 not found

So naturally, I tried to use two backward slashes. Sadly, this did not work:

$ winpty docker run -it sequenceiq\\hadoop-docker:2.7.0 /etc/bootstrap.sh -bash
Unable to find image 'sequenceiq\hadoop-docker:2.7.0' locally
FATA[0000] Invalid repository name (sequenceiq\hadoop-docker), only [a-z0-9-_.] are allowed

I was hopeful for a minute, but I'm sad again.

Should I be pursuing this issue on this forum or on the 'docker toolbox' forum?

[Kosta-Github]

This is not docker specific, e.g.:

$ winpty echo http://google.de http://google.de
http:\google.de http:\google.de

But I couldn't find out yet, if the winpty bundled with the docker toolbox is behaving differently than the official winpty command...

[Kosta-Github]

And I haven't found a way to avoid that conversion... :-(

[medined]

I don't have access to my terminal now, but someone suggesting using ^
(caret) as an escape character. I'll try this tonight.

On Fri, Nov 6, 2015 at 10:34 AM, Kosta notifications@github.com wrote:

This is not docker specific, e.g.:

$ winpty echo http://google.de
http:\google.de

But I couldn't find out yet, if the winpty bundled with the docker toolbox
is behaving differently than the official winpty command...

—
Reply to this email directly or view it on GitHub
#56 (comment).

[rprichard]

I wouldn't expect ^ to help. That's the cmd.exe escape character, but cmd.exe isn't involved here.

I guessed correctly -- the extra conversion was added for the MSYS2 packaging: https://github.com/Alexpux/MSYS2-packages/blob/master/winpty-git/0002-Add-support-for-conversion-of-all-argv-path-args.patch

Edited to add: It's going to be impossible to avoid the slash conversion with the existing winpty.exe binary. We'll need to modify winpty.exe somehow. Either it should do no conversion (i.e. the MSYS2 patch should be removed), or it should do something closer to what MSYS2 does.

[rprichard]

For MSYS2, the function winpty needs is arg_heuristic / convert:

• https://github.com/Alexpux/Cygwin/blob/d6cd1b3aa83f0c57e0c47771f6c0b33fb8a90047/winsup/cygwin/spawn.cc#L405
• https://github.com/Alexpux/Cygwin/blob/d6cd1b3aa83f0c57e0c47771f6c0b33fb8a90047/winsup/cygwin/path.cc#L3476
• https://github.com/Alexpux/Cygwin/blob/d6cd1b3aa83f0c57e0c47771f6c0b33fb8a90047/winsup/cygwin/msys2_path_conv.cc#L261

Curiously, the arg_heuristic function is declared in the /usr/include/sys/cygwin.h header, so I can compile a C++ file that calls it, but it's not exported from the MSYS-2.0.dll, so the program won't link.

Alternatively, when the winpty console.exe interface invokes the child process, it could somehow use the POSIX fork/exec mechanism. Perhaps the agent could optionally be compiled as a POSIX program. Perhaps console.exe could (1) fork, then (2) attach to the new console, then (3) exec the child process. My concern is that, while I can understand the POSIX and Windows APIs individually, I'm a little wary of combining them.

[medined]

I tried to use the ^ character as an escape just for completeness. And yes, it did not help.

I'd be happy to help resolve this issue if there was something I can do. If you want me to run a debug version of winpty I'd be happy to try.

Unfortunately I don't have the knowledge to help with the coding.

[rprichard]

For the time being, I think you could build this winpty repository for MSYS2 and run console.exe instead of the MSYS2-packaged winpty.exe. (I don't have up-to-date binaries published right now.)

[medined]

I got the idea to rename (actually retag) the image that I'm trying to use:

docker tag sequenceiq/hadoop-docker hadoop-docker

Then I tried to use that new name. And got a different error:

$ winpty docker run -it hadoop-docker /etc/bootstrap.sh -bash
exec: "C:\\Program Files\\Git\\etc\\bootstrap.sh": executable file not found in $PATH
FATA[0000] Error response from daemon: Cannot start container c50329ebbf204223404175733e16c44e6ef6043c722d1962e1ad98384ad1cd7e: [8] System error: exec: "C:\\Program Files\\Git\\etc\\bootstrap.sh": executable file not found in $PATH

[rprichard]

It's the same problem. /etc/bootstrap.sh is translated to C:\Program Files\Git\etc\bootstrap.sh.

[medined]

I have a workaround. A hack. But it's enough to close this ticket.

1. re-tag the image to avoid the slash character.
2. start the container using "winpty docker run -it hadoop-docker bash"
3. execute the bootstrap script using "/etc/bootstrap.sh bash"

Now hadoop is running.

Thanks for helping me to understand the issues.