Action re-run fails when action contains a secret parameter

[sidkrishna]

Re-runs for actions that contain a secret param eg. password fail both from CLI and UI. This is happening most probably because the parameter gets passed as ****** instead of the actual value.
If one changes the param from **** to actual text, it works.

Logs:

brocade@BWC:/opt/stackstorm/packs$ st2 execution list -n 1
+--------------------------+-----------------------+--------------+------------------------+-----------------------+--------------------+
| id                       | action.ref            | context.user | status                 | start_timestamp       | end_timestamp      |
+--------------------------+-----------------------+--------------+------------------------+-----------------------+--------------------+
| 58e605f2c3540c086edb46b6 | network_essentials.ge | st2admin     | succeeded (2s elapsed) | Thu, 06 Apr 2017      | Thu, 06 Apr 2017   |
|                          | t_switch_details      |              |                        | 09:10:10 UTC          | 09:10:12 UTC       |
+--------------------------+-----------------------+--------------+------------------------+-----------------------+--------------------+
brocade@BWC:/opt/stackstorm/packs$ st2 execution re-run 58e605f2c3540c086edb46b6
...
id: 58e60699c3540c086edb46b8
status: failed
parameters:
  mgmt_ip: 10.26.7.223
  password: '********'
  username: admin
result:
  exit_code: 1
  result: None
  stderr: "Traceback (most recent call last):
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py", line 242, in <module>
    obj.run()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py", line 154, in run
    output = action.run(**self._parameters)
  File "/opt/stackstorm/packs/network_essentials/actions/get_switch_details.py", line 36, in run
    raise ValueError(e)
ValueError: AuthenticationException('Authentication failed.',)
"
  stdout: ''

Here, I am re-running the first action which completed successfully earlier but the re-run fails with authentication failure. The same behavior is seen from the UI. I've tested this with more actions/workflows as well.

[enykeev]

I can confirm the issue and in some cases it might lead to truly awful consequences. This is the action I was using for testing:

---
description: Action that executes an arbitrary Linux command on the localhost.
enabled: true
entry_point: ''
name: some
parameters:
  stuff:
    type: string
    secret: true
  cmd:
    description: Arbitrary Linux command to be executed on the local host.
    required: true
    type: string
    immutable: true
    default: echo {{ stuff }}
  sudo:
    immutable: true
runner_type: "local-shell-cmd"

and this is log of running the action

vagrant@st2packages-u14:~$ st2 run default.some stuff=thing
.
id: 58e709a11878c11a997bf4e7
status: succeeded
parameters:
  stuff: '********'
result:
  failed: false
  return_code: 0
  stderr: ''
  stdout: thing
  succeeded: true
vagrant@st2packages-u14:~$ st2 execution re-run 58e709a11878c11a997bf4e7
.
id: 58e709cc1878c11a997bf4ea
status: succeeded
parameters:
  stuff: '********'
result:
  failed: false
  return_code: 0
  stderr: ''
  stdout: bin boot dev etc home initrd.img lib lib64 lost+found media mnt opt proc root run sbin srv sys tmp usr vagrant var vmlinuz
  succeeded: true

Now imagine it's not echo, but rm -Rf.

[lakshmi-kannan]

@enykeev @sidkrishna I tried to repro and I fail to repro. Looking at the new API controller code for re-run, I don't see that this bug can happen.

(virtualenv)vagrant@st2dev /m/s/s/st2 ❯❯❯ st2 run examples.momma_secrets stuff=covfefe                  issue_3339/rerun_with_secrets ✭ ✱ ◼
.
id: 594166c5d9d7ed59c8a2b3dd
status: succeeded
parameters:
  stuff: '********'
result:
  failed: false
  return_code: 0
  stderr: ''
  stdout: covfefe
  succeeded: true
(virtualenv)vagrant@st2dev /m/s/s/st2 ❯❯❯ st2 execution re-run 594166c5d9d7ed59c8a2b3dd                 issue_3339/rerun_with_secrets ✭ ✱ ◼
.
id: 594166d6d9d7ed59c8a2b3e0
status: succeeded
parameters:
  stuff: '********'
result:
  failed: false
  return_code: 0
  stderr: ''
  stdout: covfefe
  succeeded: true
(virtualenv)vagrant@st2dev /m/s/s/st2 ❯❯❯ st2 action get examples.momma_secrets                         issue_3339/rerun_with_secrets ✭ ✱ ◼
+-------------+--------------------------------------------------------------+
| Property    | Value                                                        |
+-------------+--------------------------------------------------------------+
| id          | 59416620d9d7ed59a127a94d                                     |
| uid         | action:examples:momma_secrets                                |
| ref         | examples.momma_secrets                                       |
| pack        | examples                                                     |
| name        | momma_secrets                                                |
| description | Action that executes an arbitrary Linux command on the       |
|             | localhost.                                                   |
| enabled     | True                                                         |
| entry_point |                                                              |
| runner_type | local-shell-cmd                                              |
| parameters  | {                                                            |
|             |     "cmd": {                                                 |
|             |         "default": "echo {{ stuff }}",                       |
|             |         "required": true,                                    |
|             |         "type": "string",                                    |
|             |         "description": "Arbitrary Linux command to be        |
|             | executed on the local host.",                                |
|             |         "immutable": true                                    |
|             |     },                                                       |
|             |     "sudo": {                                                |
|             |         "immutable": true                                    |
|             |     },                                                       |
|             |     "stuff": {                                               |
|             |         "secret": true,                                      |
|             |         "type": "string"                                     |
|             |     }                                                        |
|             | }                                                            |
| notify      |                                                              |
| tags        |                                                              |
+-------------+--------------------------------------------------------------+
(virtualenv)vagrant@st2dev /m/s/s/st2 ❯❯❯

Closing issue. If you guys think there is still a problem, reopen please. st2 version should be >= 2.3.0. This is the current unstable release which is going to become stable soon.

[sidkrishna]

@lakshmi-kannan I am still facing the issue on the latest unstable version:

brocade@testvm:/opt/stackstorm/packs/vslx/actions/workflows$ st2 --version
st2 2.3dev


brocade@testvm:/opt/stackstorm/packs/vslx/actions/workflows$ st2 execution list -n 1
+----------------------------+-------------------------+--------------+-----------------------+-------------------------+---------------+
| id                         | action.ref              | context.user | status                | start_timestamp         | end_timestamp |
+----------------------------+-------------------------+--------------+-----------------------+-------------------------+---------------+
| + 59426302038c432d63787460 | vslx.vslx_create_chassi | st2admin     | running (46s elapsed) | Thu, 15 Jun 2017        |               |
|                            | s                       |              |                       | 10:35:46 UTC            |               |
+----------------------------+-------------------------+--------------+-----------------------+-------------------------+---------------+
brocade@testvm:/opt/stackstorm/packs/vslx/actions/workflows$ st2 execution get 59426302038c432d63787460
id: 59426302038c432d63787460
action.ref: vslx.vslx_create_chassis
parameters:
  chassis_name: test_ch
  chassis_type: 1U
  host_ip: 10.26.7.251
  host_pwd: '********'
  host_usr: brocade
  template_name: temp_9540
status: running (54s elapsed)
start_timestamp: 2017-06-15T10:35:46.054695Z
end_timestamp: None
+-----------------------------+------------------------+------------------------------+------------------------------+-------------------------------+
| id                          | status                 | task                         | action                       | start_timestamp               |
+-----------------------------+------------------------+------------------------------+------------------------------+-------------------------------+
| + 59426302038c432d63787462  | failed (26s elapsed)   | setup_ssh                    | vslx.vslx_internal_setup_ssh | Thu, 15 Jun 2017 10:35:46 UTC |
|    59426302038c432d63787464 | succeeded (1s elapsed) | install_pwdless_ssh_dependen | core.local_sudo              | Thu, 15 Jun 2017 10:35:46 UTC |
|                             |                        | cy                           |                              |                               |
|    59426303038c432d63787466 | succeeded (0s elapsed) | set_root_known_hosts         | core.local_sudo              | Thu, 15 Jun 2017 10:35:47 UTC |
|    59426304038c432d63787468 | succeeded (0s elapsed) | set_stanley_known_hosts      | core.local                   | Thu, 15 Jun 2017 10:35:48 UTC |
|    59426304038c432d6378746a | failed (0s elapsed)    | config_pwdless_ssh           | core.local                   | Thu, 15 Jun 2017 10:35:48 UTC |
+-----------------------------+------------------------+------------------------------+------------------------------+-------------------------------+
brocade@testvm:/opt/stackstorm/packs/vslx/actions/workflows$ st2 execution get 59426304038c432d6378746a
id: 59426304038c432d6378746a
status: failed (0s elapsed)
parameters:
  cmd: sshpass -p ******** ssh-copy-id -i /home/stanley/.ssh/stanley_rsa.pub brocade@10.26.7.251
result:
  failed: true
  return_code: 3
  stderr: 'sshpass: Failed to run command: No such file or directory'
  stdout: ''
  succeeded: false
brocade@testvm:/opt/stackstorm/packs/vslx/actions/workflows$

[sidkrishna]

@lakshmi-kannan I tried this out using the UI so that might have implications vs how the CLI behaves.

[lakshmi-kannan]

@sidkrishna Can you try the simple example @enykeev posted on your box and report what happens with that? I have a feeling this might be a problem with the vlsx actions. Can you also point me to that pack?

[sidkrishna]

@lakshmi-kannan Tried it with the action as defined by @enykeev above. I issue seems to only happen when execution is re-run via the UI. Re-run from CLI is working fine:

CLI Re-run:

brocade@testvm:/opt/stackstorm/packs/default/actions$ st2 execution re-run 5943a73d038c432d6378746c
.
id: 5943a763038c432d6378746f
status: succeeded
parameters:
  stuff: '********'
result:
  failed: false
  return_code: 0
  stderr: ''
  stdout: thing
  succeeded: true

UI Re-run:

brocade@testvm:/opt/stackstorm/packs/default/actions$ st2 execution list -n 1
+--------------------------+--------------+--------------+------------------------+-------------------------+------------------------+
| id                       | action.ref   | context.user | status                 | start_timestamp         | end_timestamp          |
+--------------------------+--------------+--------------+------------------------+-------------------------+------------------------+
| 5943a7d0038c432d63787475 | default.some | st2admin     | succeeded (0s elapsed) | Fri, 16 Jun 2017        | Fri, 16 Jun 2017       |
|                          |              |              |                        | 09:41:36 UTC            | 09:41:36 UTC           |
+--------------------------+--------------+--------------+------------------------+-------------------------+------------------------+
brocade@testvm:/opt/stackstorm/packs/default/actions$ st2 execution get 5943a7d0038c432d63787475
id: 5943a7d0038c432d63787475
status: succeeded (0s elapsed)
parameters:
  stuff: '********'
result:
  failed: false
  return_code: 0
  stderr: ''
  stdout: bin boot dev etc home initrd.img lib lib64 lost+found media mnt opt proc root run sbin srv sys tmp usr var vmlinuz
  succeeded: true

[lakshmi-kannan]

Thanks for confirming @sidkrishna. I'll move this to st2web.