Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hide security-sensitive strings in VCS command log messages #6890

Merged
merged 4 commits into from Aug 21, 2019

Conversation

@cjerdonek
Copy link
Member

cjerdonek commented Aug 17, 2019

This PR adds a new HiddenText class to wrap sensitive strings and starts using them in the VCS modules to protect (1) URL's containing potential auth info, and (2) passwords that can be included in VCS command-line arguments. This finishes the work started in PR #6862.

@cjerdonek cjerdonek force-pushed the cjerdonek:vcs-hidden-text branch 3 times, most recently Aug 17, 2019
@cjerdonek cjerdonek force-pushed the cjerdonek:vcs-hidden-text branch Aug 17, 2019
Copy link
Member

chrahunt left a comment

What a nice idea, good job. :)

src/pip/_internal/utils/misc.py Show resolved Hide resolved
src/pip/_internal/vcs/bazaar.py Outdated Show resolved Hide resolved
src/pip/_internal/vcs/git.py Outdated Show resolved Hide resolved
src/pip/_internal/vcs/versioncontrol.py Show resolved Hide resolved
tests/unit/test_utils.py Show resolved Hide resolved
@cjerdonek cjerdonek force-pushed the cjerdonek:vcs-hidden-text branch to 5ffc930 Aug 18, 2019
@cjerdonek

This comment has been minimized.

Copy link
Member Author

cjerdonek commented Aug 18, 2019

PR updated. Thanks again for the thorough review, @chrahunt. 🙏

Copy link
Member

chrahunt left a comment

One minor comment, otherwise LGTM.

src/pip/_internal/vcs/subversion.py Show resolved Hide resolved
src/pip/_internal/utils/misc.py Outdated Show resolved Hide resolved
@cjerdonek

This comment has been minimized.

Copy link
Member Author

cjerdonek commented Aug 19, 2019

PR updated. I also implemented HiddenText.__ne__() (which is needed for Python 2) and added some unit tests for HiddenText's equality operators.

@cjerdonek cjerdonek merged commit 7783c47 into pypa:master Aug 21, 2019
22 checks passed
22 checks passed
Linux Build #20190818.7 succeeded
Details
Linux (Package) Package succeeded
Details
Linux (Test Primary Python27) Test Primary Python27 succeeded
Details
Linux (Test Primary Python36) Test Primary Python36 succeeded
Details
Linux (Test Secondary Python35) Test Secondary Python35 succeeded
Details
Linux (Test Secondary Python37) Test Secondary Python37 succeeded
Details
Windows Build #20190818.7 succeeded
Details
Windows (Package) Package succeeded
Details
Windows (Test Primary Python27-x86) Test Primary Python27-x86 succeeded
Details
Windows (Test Primary Python37-x64) Test Primary Python37-x64 succeeded
Details
Windows (Test Secondary Python35-x86) Test Secondary Python35-x86 succeeded
Details
Windows (Test Secondary Python36-x86) Test Secondary Python36-x86 succeeded
Details
Windows (Test Secondary Python37-x86) Test Secondary Python37-x86 succeeded
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
macOS Build #20190818.7 succeeded
Details
macOS (Package) Package succeeded
Details
macOS (Test Primary Python27) Test Primary Python27 succeeded
Details
macOS (Test Primary Python36) Test Primary Python36 succeeded
Details
macOS (Test Secondary Python35) Test Secondary Python35 succeeded
Details
macOS (Test Secondary Python37) Test Secondary Python37 succeeded
Details
news-file/pr News files updated and/or change is trivial.
Details
@cjerdonek cjerdonek deleted the cjerdonek:vcs-hidden-text branch Aug 21, 2019
@cjerdonek

This comment has been minimized.

Copy link
Member Author

cjerdonek commented Aug 21, 2019

Thanks again for the thoughtful reviews, @chrahunt and @pradyunsg!

# Also, we don't apply str() to arguments that aren't HiddenText since
# this can trigger a UnicodeDecodeError in Python 2 if the argument
# has type unicode and includes a non-ascii character. (The type
# checker doesn't ensure the annotations are correct in all cases.)

This comment has been minimized.

Copy link
@pradyunsg

pradyunsg Aug 21, 2019

Member

Reg the type-checker: I think we're supposed to use mypy.Text for things that would be unicode on Python 2.

This comment has been minimized.

Copy link
@cjerdonek

cjerdonek Aug 21, 2019

Author Member

This is supposed to always be str (if everything is working correctly), but because our type-checking isn't 100%, it's possible something of unicode type can slip through the cracks (e.g. because of # type: ignore comments at some of our annotations). One example is the _get_win_folder_with_ctypes() function in appdirs.py that is annotated with str but returns unicode in Python 2. In cases that slip through the cracks, the point was not to force a crash if we don't need to..

@pradyunsg

This comment has been minimized.

Copy link
Member

pradyunsg commented Aug 21, 2019

This PR looks great @cjerdonek! ^>^

@lock lock bot added the S: auto-locked label Sep 20, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Sep 20, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
3 participants
You can’t perform that action at this time.