Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Cli password fixin #42645

Open
wants to merge 5 commits into
base: devel
Choose a base branch
from
Open

Conversation

abadger
Copy link
Contributor

@abadger abadger commented Jul 11, 2018

SUMMARY

When prompted for passwords at the CLI, we want those passwords to pass through the system as untemplated byte strings.

ISSUE TYPE
  • Bugfix Pull Request
COMPONENT NAME
ANSIBLE VERSION
devel (2.7.0)
ADDITIONAL INFORMATION

@ansibot ansibot added WIP This issue/PR is a work in progress. Nevertheless it was shared for getting input from peers. affects_2.7 This issue/PR affects Ansible v2.7 bug This issue/PR relates to a bug. needs_triage Needs a first human triage before being processed. support:core This issue/PR relates to code supported by the Ansible Engineering Team. labels Jul 11, 2018
@ansibot
Copy link
Contributor

ansibot commented Jul 11, 2018

The test ansible-test sanity --test pylint [explain] failed with 3 errors:

lib/ansible/playbook/base.py:446:32: undefined-variable Undefined variable 'to_bytes'
lib/ansible/playbook/base.py:453:35: undefined-variable Undefined variable 'to_bytes'
lib/ansible/playbook/play_context.py:397:0: syntax-error invalid syntax (<string>, line 397)

The test ansible-test sanity --test compile --python 2.6 [explain] failed with 1 error:

lib/ansible/playbook/play_context.py:397:116: SyntaxError: new_info['become_pass'] = to_bytes(variables[sudo_pass_name], errors='surrogate_or_strict'))

The test ansible-test sanity --test compile --python 2.7 [explain] failed with 1 error:

lib/ansible/playbook/play_context.py:397:116: SyntaxError: new_info['become_pass'] = to_bytes(variables[sudo_pass_name], errors='surrogate_or_strict'))

The test ansible-test sanity --test compile --python 3.5 [explain] failed with 1 error:

lib/ansible/playbook/play_context.py:397:116: SyntaxError: new_info['become_pass'] = to_bytes(variables[sudo_pass_name], errors='surrogate_or_strict'))

The test ansible-test sanity --test compile --python 3.6 [explain] failed with 1 error:

lib/ansible/playbook/play_context.py:397:116: SyntaxError: new_info['become_pass'] = to_bytes(variables[sudo_pass_name], errors='surrogate_or_strict'))

The test ansible-test sanity --test compile --python 3.7 [explain] failed with 1 error:

lib/ansible/playbook/play_context.py:397:116: SyntaxError: new_info['become_pass'] = to_bytes(variables[sudo_pass_name], errors='surrogate_or_strict'))

The test ansible-test sanity --test docs-build [explain] failed with the error:

Command "/usr/bin/python test/sanity/code-smell/docs-build.py" returned exit status 1.
>>> Standard Error
Traceback (most recent call last):
  File "test/sanity/code-smell/docs-build.py", line 101, in <module>
    main()
  File "test/sanity/code-smell/docs-build.py", line 17, in main
    raise subprocess.CalledProcessError(sphinx.returncode, cmd, output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['make', 'singlehtmldocs']' returned non-zero exit status 2.

The test ansible-test sanity --test pep8 [explain] failed with 147 errors:

lib/ansible/playbook/play_context.py:398:25: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:399:13: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:402:48: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:402:50: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:402:91: E225 missing whitespace around operator
lib/ansible/playbook/play_context.py:405:9: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:406:9: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:407:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:407:28: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:409:9: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:410:9: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:418:34: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:418:36: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:419:35: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:419:37: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:425:9: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:426:9: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:427:9: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:428:9: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:429:9: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:430:9: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:432:41: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:432:43: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:433:33: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:433:35: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:435:9: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:436:9: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:437:28: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:437:30: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:439:9: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:440:32: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:440:34: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:442:9: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:443:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:443:28: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:445:9: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:447:5: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:450:15: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:450:17: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:451:20: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:451:22: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:452:20: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:452:22: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:457:27: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:457:29: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:459:22: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:459:24: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:460:21: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:460:23: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:461:24: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:461:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:462:24: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:462:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:465:24: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:465:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:467:24: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:467:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:470:16: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:470:18: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:473:18: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:473:20: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:488:27: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:488:29: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:489:30: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:489:32: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:491:30: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:491:32: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:497:38: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:497:40: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:499:38: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:499:40: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:500:49: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:500:51: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:502:23: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:502:25: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:504:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:504:28: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:508:23: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:508:25: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:509:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:509:28: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:515:23: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:515:25: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:516:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:516:28: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:521:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:521:28: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:527:41: E113 unexpected indentation
lib/ansible/playbook/play_context.py:528:17: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:528:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:528:28: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:530:13: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:532:23: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:532:25: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:533:20: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:533:22: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:542:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:542:28: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:544:13: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:546:20: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:546:22: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:548:27: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:548:29: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:549:30: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:549:32: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:551:30: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:551:32: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:553:13: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:555:20: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:555:22: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:557:23: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:557:25: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:558:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:558:28: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:560:13: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:562:20: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:562:22: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:563:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:563:28: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:565:13: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:568:13: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:569:28: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:569:30: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:570:13: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:570:29: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:570:31: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:571:13: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:573:9: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:575:5: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:586:24: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:586:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:589:43: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:589:45: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:593:5: E122 continuation line missing indentation or outdented
lib/ansible/playbook/play_context.py:595:18: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:595:20: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:597:22: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:597:24: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:602:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:602:28: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:606:30: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:606:32: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:610:22: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:610:24: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:613:28: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:613:30: E251 unexpected spaces around keyword / parameter equals
lib/ansible/playbook/play_context.py:616:1: E901 TokenError: EOF in multi-line statement

click here for bot help

@abadger abadger force-pushed the cli-password-fixin branch 2 times, most recently from 1396269 to 4b8f8f1 Compare July 11, 2018 23:03
We should have designed this so that post-validate and the rest of the
code here assigned semantic meaning to tokens that were parsed from the
playbook.  Unfortunately, we combined that semantic meaning with the
live playbook objects that are used by the rest of the program.  So in
post_validate we can get information that may not have come from the
playbook.

This is important because all playbook strings are text strings.  But
strings from other parts of the code may be byte strings instead.  So we
need to throw an error if we cannot convert the byte string into a text
string.  If we don't do that, we would silently mangle the string in the
convdersion and pass it on.  That would cause issues later on, for
instance, passwords that don't work with ssh or become.

Additionally, add a comment that to_text() is really here to convert
from non-string types into text.  For instance, from int and float.
If we get to the point where we are only dealing with text strings at
this point, we will still have to keep the to_text() to convert other
types.
With regular jinja,
* ascii bytes will get mangled on Python3 (text: "b'test'")
* nonascii bytes will cause a traceback in Python2

With NativeTypes jinja:
* non-ascii bytes will cause a traceback in jinja2 if it's mixed with
  text type on Python2
* any bytes will be mangled if they're mixed with text strings on
  Python3
Become and ssh pass can be read in from a password prompt.  When that
happens, we get a byte string.  Just like vault passwords, we want these
password prompts to remain byte strings all the way through the code so
that we don't lose any information from the user.
This way we can tell which scenario failed and test all three scenarios
even if one of the early ones failed.
@ansibot ansibot added the test This PR relates to tests. label Jul 12, 2018
@jborean93 jborean93 removed the needs_triage Needs a first human triage before being processed. label Jul 12, 2018
@ansibot ansibot added the stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. label Jul 20, 2018
@ansibot ansibot added the needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html label Sep 14, 2018
@ansibot ansibot added the support:community This issue/PR relates to code supported by the Ansible community. label Sep 22, 2018
@ansibot ansibot added pre_azp This PR was last tested before migration to Azure Pipelines. and removed stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. labels Dec 6, 2020
@ansibot ansibot removed the support:community This issue/PR relates to code supported by the Ansible community. label Mar 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects_2.7 This issue/PR affects Ansible v2.7 bug This issue/PR relates to a bug. needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html pre_azp This PR was last tested before migration to Azure Pipelines. support:core This issue/PR relates to code supported by the Ansible Engineering Team. test This PR relates to tests. WIP This issue/PR is a work in progress. Nevertheless it was shared for getting input from peers.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants