1159 create mirror of private GitHub repository

[eugenk]

Updated message

This fixes #1159 and a few other git/cloning related bugs (see first commits).
When a private git repository is supposed to be cloned, git usually asks for credentials. This prompt is prevented by asking /bin/true for them. Of course, true doesn't supply any, but exits with code SUCCESS, so git thinks they are supplied and fails because the credentials don't fit to the repository.

I removed the previous workaround-commits by rebase (see original message).

Original message

This is more of a workaround than a fix for #1159.

I don't see any fast way to check if something is a git repository than git ls-remote. However, this asks for credentials when calling it on a github private repository.

I tried to read the login prompt of git ls-remote in Subprocess.run, but I could not get my hands on the prompt lines. Neither with IO.popen nor with Open3.popen2e.

The best solution would be to read those login prompt lines and abort if they are present, but I can't figure out how to do this, because they are not printed to stdout nor stderr of the child process.

[houndci-bot]

Metrics/LineLength: Line is too long. [136/80]

[eugenk]

Other possibilities to take a look at include

• expect/autoexpect
• output redirection via script -q -c 'git ls-remote private_repo' > output_file (only works on linux)

[jelmd]

Not sure, what you are trying to accomplish, but "command ...| tee file" could be a posix conform aka *x compatible thing you are looking for.

[eugenk]

The thing is, that git ls-remote https://github.com/some_user/private_repository.git asks for a username and a password and I want to get my hands on the output asking for the credentials. This output is neither going to stdout not to stderr, but I need it to go to stdout/stderr. Otherwise I cannot access it from inside ruby while the process is running.

[jelmd]

Hmmm, but script just captures stdout and stderr (I guess the > is just a typo), the | stdout [and stderr if redirected], only. So if you wanna capture both in a file (and not directly from stdout,stderr), command ... >file 2>&1 redirection is sufficient. Anyway, if you say, you do not get anything via stdout/stderr than whatever of the previous things one does, one would always get an empty file.

So the only thing what would explain that, is IMHO, that the related fds get closed on exec - possible, but I doubt that. Have you a conrete working snippet, which I can use for playing around ...?

[tillmo]

"neither stdout nor stderr" here means the usual practice of many commands that the password prompt is directly sent to the tty, bypassing stdout/stderr and any redirections of these. Commands like expect redirect this using a pty.

[eugenk]

You can directly try git ls-remote https://github.com/some_user/private_repository.git (yes, with "some_user"). This prompts for credentials.

FYI: I used > to illustrate the redirection. In ruby, I call either IO.popen or popen2e to spawn a process and read from stdout and stderr. There are no shell builtins accessible like > or |.

I successfully tried script -q -c ... on an arch linux and it put the output into a file. This is not a good solution, though, because it's only working on linux and not on osx (some developers here use osx).

[jelmd]

What about abusing GIT_ASKPSS? E.g.:

echo '#!/bin/sh\nprint "$@" >/tmp/out.txt' >/tmp/ask.sh
chmod 755 /tmp/ask.sh
export GIT_ASKPSS=/tmp/ask.sh
git ls-remote ...

Fore more see http://code.metager.de/source/xref/git/prompt.c#44 ;-)

[eugenk]

The script as you posted does not work for me. However, it's a very good pointer to GIT_ASKPASS. Thanks!

This does exactly what I'm looking for (because we don't want to use credentials at this point):

GIT_ASKPASS=echo git ls-remote https://github.com/some_user/private_repository.git

[eugenk]

Now the pull request is ready for review. In my opinion, it's a fix for the problem and not a workaround anymore.

[0robustus1]

You could do the following instead:

self.source_type =
  if GitRepository.is_git_repository?(source_address)
    'git'
  ....

[eugenk]

I ignore the hound comments after the last commit because they don't match to our style guide (and hound has the hiccups).

[0robustus1]

👍