Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redact unsafe URLs in the Trace2 output #616

Merged
merged 5 commits into from
Nov 8, 2023

Conversation

dscho
Copy link
Member

@dscho dscho commented Nov 5, 2023

The Trace2 output can contain secrets when a user issues a Git command with sensitive information in the command-line. A typical (if highly discouraged) example is: git clone https://user:password@host.com/.

With this PR, the Trace2 output redacts passwords in such URLs by default.

@dscho dscho self-assigned this Nov 5, 2023
@jeffhostetler
Copy link
Collaborator

Unrelated to Trace2, the same problem can be seen in GIT_TRACE_PERFORMANCE:

% GIT_TRACE_PERFORMANCE=1  ../../git clone https://user:pw@github.com/jeffhostetler/dotfiles d2 
Cloning into 'd2'...
warning: templates not found in /Users/jeff/share/git-core/templates
git: 'remote-https' is not a git command. See 'git --help'.
13:53:40.538006 trace.c:414             performance: 0.003359000 s: git command: /opt/homebrew/bin/git remote-https origin https://user:pw@github.com/jeffhostetler/dotfiles
13:53:40.540409 trace.c:414             performance: 0.019898000 s: git command: ../../git clone https://user:pw@github.com/jeffhostetler/dotfiles d2
m1 ~/work/microsoft_git.git/t/trash directory.t0210-trace2-normal % 

dscho and others added 3 commits November 6, 2023 17:02
It is an unsafe practice to call something like

	git clone https://user:password@example.com/

This not only risks leaking the password "over the shoulder" or into the
readline history of the current Unix shell, it also gets logged via
Trace2 if enabled.

Let's at least avoid logging such secrets via Trace2, much like we avoid
logging secrets in `http.c`. Much like the code in `http.c` is guarded
via `GIT_TRACE_REDACT` (defaulting to `true`), we guard the new code via
`GIT_TRACE2_REDACT` (also defaulting to `true`).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Add `struct key_value_info` argument to `trace2_def_param()`.

In dc90208 (trace2: plumb config kvi, 2023-06-28) a `kvi`
argument was added to `trace2_def_param_fl()` but the macro
was not up updated. Let's fix that.

Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Turn off TEST_PASSES_SANITIZE_LEAK in t0210 and t0211 tests.

The tests added in a previous commit to confirm that we redact URLs in
the Trace2 output uncovered leaks in `builtin/clone.c` and `remote.c`.

We observed that (1) `the_repository->remote_status` is not released
properly.

And (2) that we are using `url...insteadOf` and that runs into a code
path where an allocated URL is replaced with another URL.

And (3) `remote_states` contains plenty of `struct remote`s whose
refspecs seem to be usually allocated by never released.

More investigation is needed here to identify the exact cause and
proper fixes these leaks / bugs.

Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
@jeffhostetler
Copy link
Collaborator

I cancelled the job because the osx-clang job seemed to be stuck after almost 90 minutes.

@jeffhostetler
Copy link
Collaborator

@dscho Should we update the workflow to eliminate the osx-gcc compile since we're now using osx-clang and universal builds. Wondering if it is completely redundant now.

@dscho
Copy link
Member Author

dscho commented Nov 7, 2023

@dscho Should we update the workflow to eliminate the osx-gcc compile since we're now using osx-clang and universal builds. Wondering if it is completely redundant now.

I'd love to keep it because upstream Git keeps it.

@jeffhostetler jeffhostetler merged commit 289cef5 into tentative/vfs-2.43.0 Nov 8, 2023
79 checks passed
@jeffhostetler jeffhostetler deleted the redact-unsafe-urls-in-trace2 branch November 8, 2023 14:25
@jeffhostetler
Copy link
Collaborator

Keeping the osc-gcc job is fine.

dscho pushed a commit that referenced this pull request Nov 8, 2023
The Trace2 output can contain secrets when a user issues a Git command
with sensitive information in the command-line. A typical (if highly
discouraged) example is: `git clone https://user:password@host.com/`.

With this PR, the Trace2 output redacts passwords in such URLs by
default.

This series also includes a commit to temporarily disable leak checking on t0210,t0211 because the tests uncover other unrelated bugs in Git.
dscho pushed a commit that referenced this pull request Nov 14, 2023
The Trace2 output can contain secrets when a user issues a Git command
with sensitive information in the command-line. A typical (if highly
discouraged) example is: `git clone https://user:password@host.com/`.

With this PR, the Trace2 output redacts passwords in such URLs by
default.

This series also includes a commit to temporarily disable leak checking on t0210,t0211 because the tests uncover other unrelated bugs in Git.
dscho pushed a commit that referenced this pull request Nov 20, 2023
The Trace2 output can contain secrets when a user issues a Git command
with sensitive information in the command-line. A typical (if highly
discouraged) example is: `git clone https://user:password@host.com/`.

With this PR, the Trace2 output redacts passwords in such URLs by
default.

This series also includes a commit to temporarily disable leak checking on t0210,t0211 because the tests uncover other unrelated bugs in Git.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants