ansible-vault edit and encrypt output different payload format version #30575

colinjc · 2017-09-19T20:10:58Z

ISSUE TYPE

Bug Report

COMPONENT NAME

ansible-vault

ANSIBLE VERSION

ansible 2.4.0.0

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

When I encrypt a file with ansible-vault encrypt vault_name --vault-id staging@vault_pass it creates a file with the header
$ANSIBLE_VAULT;1.2;AES256;staging
When editing a vault file with ansible-vault edit vault_name --vault-id staging@vault_pass the vault is outputting with a header of
$ANSIBLE_VAULT;1.1;AES256
If I have made any changes and try to edit the file again, the following error occurs.
ERROR! Unexpected Exception, this is probably a bug: list index out of range
There is no error if I make no changes.

If I run decrypt it is successful. If I then run
ansible-vault encrypt vault_name --vault_id staging@vault_pass
the vault is outputting with a header of $ANSIBLE_VAULT;1.2;AES256;staging and I can run ansible-deploy edit again successfully.

STEPS TO REPRODUCE

ansible-vault edit vault_name --vault-id staging@vault_pass
ansible-vault edit vault_name --vault-id staging@vault_pass
ansible-vault decrypt vault_name --vault_id staging@vault_pass
ansible-vault encrypt vault_name --vault_id staging@vault_pass

EXPECTED RESULTS

ACTUAL RESULTS

$ head -n1 vault
$ANSIBLE_VAULT;1.2;AES256;staging
$ ansible-vault edit vault_name --vault_id staging@vault_pass
$ head -n1 group_vars/staging/vault
$ANSIBLE_VAULT;1.1;AES256
$ ansible-vault edit vault_name --vault_id staging@vault_pass
$ ERROR! Unexpected Exception, this is probably a bug: list index out of range
ansible-vault decrypt vault_name --vault_id staging@vault_pass
ansible-vault encrypt vault_name --vault_id staging@vault_pass
$ head -n1 vault
$ANSIBLE_VAULT;1.2;AES256;staging

The text was updated successfully, but these errors were encountered:

alikins · 2017-09-22T14:42:39Z

Is there an 'vault_password_file' or 'vault_identity_list' setup in configuration?

Running 'ansible-vault' with a high verbosity level will show some details about which vault-ids are being used.

alikins · 2017-09-22T14:51:20Z

I can reproduce this:

[newswoop:F25:ansible (facts_arch_linux_30600 %)]$ echo '30575' > vault_30575.yml

[newswoop:F25:ansible (facts_arch_linux_30600 %)]$ ansible-vault encrypt -vvv --vault-id staging@password vault_30575.yml 
ansible-vault 2.5.0 (facts_arch_linux_30600 08cca42fd4) last updated 2017/09/22 10:22:26 (GMT -400)
  config file = /home/adrian/src/ansible/ansible.cfg
  configured module search path = [u'/home/adrian/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/adrian/src/ansible/lib/ansible
  executable location = /home/adrian/src/ansible/bin/ansible-vault
  python version = 2.7.13 (default, May 10 2017, 20:04:28) [GCC 6.3.1 20161221 (Red Hat 6.3.1-1)]
Using /home/adrian/src/ansible/ansible.cfg as config file
Encryption successful

[newswoop:F25:ansible (facts_arch_linux_30600 %)]$ cat vault_30575.yml 
$ANSIBLE_VAULT;1.2;AES256;staging
<..>

vault edit without any changes

[newswoop:F25:ansible (facts_arch_linux_30600 %)]$ ansible-vault edit -vvvvvv --vault-id staging@password vault_30575.yml 
< version info snipped >
Using /home/adrian/src/ansible/ansible.cfg as config file
Reading vault password file: password
Found a vault_id (staging) in the vaulttext
We have a secret associated with vault id (staging), will try to use to decrypt None
Trying to use vault secret=(FileVaultSecret(filename='/home/adrian/src/ansible/password')) id=staging to decrypt None
Trying secret FileVaultSecret(filename='/home/adrian/src/ansible/password') for vault_id=staging
decrypt succesful with secret=FileVaultSecret(filename='/home/adrian/src/ansible/password') and vault_id=staging

[newswoop:F25:ansible (facts_arch_linux_30600 %)]$ cat vault_30575.yml
$ANSIBLE_VAULT;1.2;AES256;staging
<..>

vault edit, making and saving changes:

[newswoop:F25:ansible (facts_arch_linux_30600 %)]$ ansible-vault edit -vvvvvv --vault-id staging@password vault_30575.yml 
< version info snipped >

Using /home/adrian/src/ansible/ansible.cfg as config file
Reading vault password file: password
Found a vault_id (staging) in the vaulttext
We have a secret associated with vault id (staging), will try to use to decrypt None
Trying to use vault secret=(FileVaultSecret(filename='/home/adrian/src/ansible/password')) id=staging to decrypt None
Trying secret FileVaultSecret(filename='/home/adrian/src/ansible/password') for vault_id=staging
decrypt succesful with secret=FileVaultSecret(filename='/home/adrian/src/ansible/password') and vault_id=staging
Encrypting with vault secret FileVaultSecret(filename='/home/adrian/src/ansible/password')

[newswoop:F25:ansible (facts_arch_linux_30600 %)]$ cat vault_30575.yml
$ANSIBLE_VAULT;1.1;AES256
<..>

vault edit again:

[newswoop:F25:ansible (facts_arch_linux_30600 %)]$ ansible-vault edit -vvvvvv --vault-id staging@password vault_30575.yml 
< version info snipped >
Using /home/adrian/src/ansible/ansible.cfg as config file
Reading vault password file: password
Found a vault_id (default) in the vaulttext
Found a vault_id (default) in the vault text, but we do not have a associated secret (--vault-id)
Trying to use vault secret=(FileVaultSecret(filename='/home/adrian/src/ansible/password')) id=staging to decrypt None
Trying secret FileVaultSecret(filename='/home/adrian/src/ansible/password') for vault_id=staging
decrypt succesful with secret=FileVaultSecret(filename='/home/adrian/src/ansible/password') and vault_id=staging
ERROR! Unexpected Exception, this is probably a bug: list index out of range
the full traceback was:

Traceback (most recent call last):
  File "/home/adrian/src/ansible/bin/ansible-vault", line 109, in <module>
    exit_code = cli.run()
  File "/home/adrian/src/ansible/lib/ansible/cli/vault.py", line 228, in run
    self.execute()
  File "/home/adrian/src/ansible/lib/ansible/cli/__init__.py", line 154, in execute
    fn()
  File "/home/adrian/src/ansible/lib/ansible/cli/vault.py", line 419, in execute_edit
    self.editor.edit_file(f)
  File "/home/adrian/src/ansible/lib/ansible/parsing/vault/__init__.py", line 768, in edit_file
    secret = secrets[0][1]
IndexError: list index out of range

alikins · 2017-09-22T14:57:50Z

@colinjc Thanks for the great bug report btw! Looking into a fix now.

alikins · 2017-09-22T15:11:05Z

Looks like the vault_id is getting dropped on the encrypt after edit with change. (vault_id isnt passed to encrypt correctly there so got a fix for that).

The traceback is from an assumption in edit that if we can decrypt, then we have a matching vault-id in the secrets. Because of above bug, on the second edit, the vault id from the edited file is unset (or 'default') but the only vault-id in secrets is 'staging'. Then the vault-id match returns a empty secrets list, and the the secrets[0][1] reference throws the IndexError.

alikins · 2017-09-22T18:21:29Z

#30772 should fix this if you want to try it out.

* Use vault_id when encrypted via vault-edit On the encryption stage of 'ansible-vault edit --vault-id=someid@passfile somefile', the vault id was not being passed to encrypt() so the files were always saved with the default vault id in the 1.1 version format. When trying to edit that file a second time, also with a --vault-id, the file would be decrypted with the secret associated with the provided vault-id, but since the encrypted file had no vault id in the envelope there would be no match for 'default' secrets. (Only the --vault-id was included in the potential matches, so the vault id actually used to decrypt was not). If that list was empty, there would be an IndexError when trying to encrypted the changed file. This would result in the displayed error: ERROR! Unexpected Exception, this is probably a bug: list index out of range Fix is two parts: 1) use the vault id when encrypting from edit 2) when matching the secret to use for encrypting after edit, include the vault id that was used for decryption and not just the vault id (or lack of vault id) from the envelope. add unit tests for #30575 and intg tests for 'ansible-vault edit' Fixes #30575 (cherry picked from commit a14d0f3)

alikins · 2017-09-26T16:33:42Z

Cherry-picked to stable-2.4 in 042079a (chery-pick of a14d0f3 from #30772)

* Use vault_id when encrypted via vault-edit On the encryption stage of 'ansible-vault edit --vault-id=someid@passfile somefile', the vault id was not being passed to encrypt() so the files were always saved with the default vault id in the 1.1 version format. When trying to edit that file a second time, also with a --vault-id, the file would be decrypted with the secret associated with the provided vault-id, but since the encrypted file had no vault id in the envelope there would be no match for 'default' secrets. (Only the --vault-id was included in the potential matches, so the vault id actually used to decrypt was not). If that list was empty, there would be an IndexError when trying to encrypted the changed file. This would result in the displayed error: ERROR! Unexpected Exception, this is probably a bug: list index out of range Fix is two parts: 1) use the vault id when encrypting from edit 2) when matching the secret to use for encrypting after edit, include the vault id that was used for decryption and not just the vault id (or lack of vault id) from the envelope. add unit tests for #30575 and intg tests for 'ansible-vault edit' Fixes #30575

* Use vault_id when encrypted via vault-edit On the encryption stage of 'ansible-vault edit --vault-id=someid@passfile somefile', the vault id was not being passed to encrypt() so the files were always saved with the default vault id in the 1.1 version format. When trying to edit that file a second time, also with a --vault-id, the file would be decrypted with the secret associated with the provided vault-id, but since the encrypted file had no vault id in the envelope there would be no match for 'default' secrets. (Only the --vault-id was included in the potential matches, so the vault id actually used to decrypt was not). If that list was empty, there would be an IndexError when trying to encrypted the changed file. This would result in the displayed error: ERROR! Unexpected Exception, this is probably a bug: list index out of range Fix is two parts: 1) use the vault id when encrypting from edit 2) when matching the secret to use for encrypting after edit, include the vault id that was used for decryption and not just the vault id (or lack of vault id) from the envelope. add unit tests for ansible#30575 and intg tests for 'ansible-vault edit' Fixes ansible#30575

ansibot added affects_2.4 This issue/PR affects Ansible v2.4 bug_report needs_triage Needs a first human triage before being processed. support:core This issue/PR relates to code supported by the Ansible Engineering Team. labels Sep 19, 2017

jborean93 removed the needs_triage Needs a first human triage before being processed. label Sep 21, 2017

alikins self-assigned this Sep 22, 2017

alikins mentioned this issue Sep 22, 2017

Use vault_id when encrypted via vault-edit #30772

Merged

alikins closed this as completed in #30772 Sep 26, 2017

alikins added the cherrypick_candidate label Sep 26, 2017

alikins added the affects_2.5 This issue/PR affects Ansible v2.5 label Sep 26, 2017

pyup-bot mentioned this issue Jan 1, 2018

Scheduled monthly dependency update for January napalm255/ansible-vmware_inventory#11

Closed

ansibot added bug This issue/PR relates to a bug. and removed bug_report labels Mar 7, 2018

ansible locked and limited conversation to collaborators Apr 26, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ansible-vault edit and encrypt output different payload format version #30575

ansible-vault edit and encrypt output different payload format version #30575

colinjc commented Sep 19, 2017

alikins commented Sep 22, 2017

alikins commented Sep 22, 2017

alikins commented Sep 22, 2017

alikins commented Sep 22, 2017

alikins commented Sep 22, 2017

alikins commented Sep 26, 2017

ansible-vault edit and encrypt output different payload format version #30575

ansible-vault edit and encrypt output different payload format version #30575

Comments

colinjc commented Sep 19, 2017

ISSUE TYPE

COMPONENT NAME

ANSIBLE VERSION

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

alikins commented Sep 22, 2017

alikins commented Sep 22, 2017

alikins commented Sep 22, 2017

alikins commented Sep 22, 2017

alikins commented Sep 22, 2017

alikins commented Sep 26, 2017