Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3000.1 - Converting module.run mine.send to new module format #56584

Closed
tazaki opened this issue Apr 8, 2020 · 10 comments · Fixed by #56635
Closed

3000.1 - Converting module.run mine.send to new module format #56584

tazaki opened this issue Apr 8, 2020 · 10 comments · Fixed by #56635
Assignees
Labels
Bug broken, incorrect, or confusing behavior Regression The issue is a bug that breaks functionality known to work in previous releases. severity-high 2nd top severity, seen by most users, causes major problems ZRelease-Sodium retired label
Projects
Milestone

Comments

@tazaki
Copy link

tazaki commented Apr 8, 2020

Description of Issue

I am unable to fix the syntax for a module.run state that is running a mine.send function.

Setup

Trying to run the following (legacy?) module.run in a state:

vault init_secrets mine:
  module.run:
  - name: mine.send
  - func: vault_init_secrets
  - kwargs:
      mine_function: vault_helper.get_init_secrets
  - args:
    - /root/init_secrets.yml

Returns the following error:

[root@saltmaster salt_formulas]# salt minion1 state.sls init_secrets_mine
minion1:
----------
          ID: vault init_secrets mine
    Function: module.run
        Name: mine.send
      Result: False
     Comment: The following arguments are missing: m_name
     Started: 17:21:10.109196
    Duration: 2.33 ms
     Changes:

Summary for minion1
------------
Succeeded: 0
Failed:    1
------------
Total states run:     1
Total run time:   2.330 ms

Note: this works pre version 3000 as shown in first comment.

Steps to Reproduce Issue

According to the docs for module.run here: https://docs.saltstack.com/en/latest/ref/states/all/salt.states.module.html

Changing to the following should work?

vault init_secrets mine:
  module.run:
  - mine.send:
    - func: vault_init_secrets
    - mine_function: vault_helper.get_init_secrets
    - /root/init_secrets.yml

But I get the following:

[root@saltmaster salt_formulas]# salt minion1 state.sls init_secrets_mine
minion1:
----------
          ID: vault init_secrets mine
    Function: module.run
      Result: False
     Comment: Module function vault init_secrets mine is not available
     Started: 17:23:59.380125
    Duration: 1.526 ms
     Changes:

Summary for minion1
------------
Succeeded: 0
Failed:    1
------------
Total states run:     1
Total run time:   1.526 ms

I can't seem to change the error no matter how I swap around the parameters. Please help, I am not sure what I am doing wrong?

Versions Report

Salt Version:
           Salt: 3000.1

Dependency Versions:
           cffi: 1.14.0
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.8.1
        libgit2: Not Installed
       M2Crypto: 0.35.2
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.6.2
   mysql-python: Not Installed
      pycparser: 2.20
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.6.8 (default, Aug  7 2019, 17:28:10)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 15.3.0
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.1.4

System Versions:
           dist: centos 7.6.1810 Core
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-957.12.2.el7.x86_64
         system: Linux
        version: CentOS Linux 7.6.1810 Core
@tazaki tazaki changed the title Converting module.run mine.send to new module format 3000.1 - Converting module.run mine.send to new module format Apr 8, 2020
@tazaki
Copy link
Author

tazaki commented Apr 8, 2020

Just wanted to verify it works on 2019.2.3 as expected:

[root@saltmaster pki]# cat init_secrets_mine.sls
vault init_secrets mine:
  module.run:
  - name: mine.send
  - func: vault_init_secrets
  - kwargs:
      mine_function: vault_helper.get_init_secrets
  - args:
    - /root/init_secrets.yml
[root@saltmaster pki]# salt minion1 state.sls vault.pki.init_secrets_mine
minion1:
----------
          ID: vault init_secrets mine
    Function: module.run
        Name: mine.send
      Result: True
     Comment: Module function mine.send executed
     Started: 23:39:33.742424
    Duration: 505.461 ms
     Changes:
              ----------
              ret:
                  True

Summary for minion1
------------
Succeeded: 1 (changed=1)
Failed:    0
------------
Total states run:     1
Total run time: 505.461 ms

Versions report:

[root@saltmaster pki]# salt --versions
Salt Version:
           Salt: 2019.2.3

Dependency Versions:
           cffi: 1.14.0
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.20
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.5 (default, Apr  9 2019, 14:30:50)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.3.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: centos 7.6.1810 Core
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-957.12.2.el7.x86_64
         system: Linux
        version: CentOS Linux 7.6.1810 Core

@tazaki
Copy link
Author

tazaki commented Apr 13, 2020

After more testing I got my module with mind.send working by adding a default value in my custom execution module.

This now leads be to believe there is a bug with passing arguments (non-key word) into the mine.send function. Here's an example:

Showing module works from commandline:

[root@saltmaster salt]# salt saltmaster file.read /opt/certs/fakeca.crt
saltmaster:
    -----BEGIN CERTIFICATE-----
    MIIEbTCCA1WgAwIBAgIYMC4zNzMwMzkxNTg2Nzk1ODk0XzdtAZjmMA0GCSqGSIb3
    DQEBCwUAMHAxCzAJBgNVBAYTAlVTMREwDwYDVQQIDAhNYXJ5bGFuZDESMBAGA1UE
    BwwJQmFsdGltb3JlMQ8wDQYDVQQKDAZTdW5heXUxDzANBgNVBAMMBmZha2VjYTEY
    MBYGCSqGSIb3DQEJARYJbm9Abm8uY29tMB4XDTIwMDQxMzE2MzgxNFoXDTIxMDQx
    MzE2MzgxNFowcDELMAkGA1UEBhMCVVMxETAPBgNVBAgMCE1hcnlsYW5kMRIwEAYD
    VQQHDAlCYWx0aW1vcmUxDzANBgNVBAoMBlN1bmF5dTEPMA0GA1UEAwwGZmFrZWNh
    MRgwFgYJKoZIhvcNAQkBFglub0Buby5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IB
    DwAwggEKAoIBAQDJ/ih/h8NkLWa1hO2MtFyoLoa5woVWm8cZd7iUtSzk+fECurw9
    K9tYudHGBh4jgvftQn6Pk0j2Oqm1efzYFbHK4/tWVGEtV05bVjOPVZdU3QIpKOtk
    ydEaVrzSn+d9TB8uRTFetLKbC/A+tpZr9mJ/CRh4ETCm5y3gYELS/ciHAQt9+Nep
    BRCRBRZzsT7+LRAxwXyDgZwftjK8mtUF12xwtEDd/OYNV2QjBQDKsCafOW27fKeu
    7lpQMiM9WEVilzf3zafEqsNH4bPkMSfYbUWX8c3io2BIzfhT80F9qb747cYts/ko
    FUvqh4jvbOE/Z9WpMX3Dl3hrDtx1542fvExNAgMBAAGjgfowgfcwEgYDVR0TAQH/
    BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAQYwHQYDVR0OBBYEFP/XcjGriiZYLbYH
    genbPqArCvI6MIGxBgNVHSMEgakwgaaAFP/XcjGriiZYLbYHgenbPqArCvI6oXSk
    cjBwMQswCQYDVQQGEwJVUzERMA8GA1UECAwITWFyeWxhbmQxEjAQBgNVBAcMCUJh
    bHRpbW9yZTEPMA0GA1UECgwGU3VuYXl1MQ8wDQYDVQQDDAZmYWtlY2ExGDAWBgkq
    hkiG9w0BCQEWCW5vQG5vLmNvbYIYMC4zNzMwMzkxNTg2Nzk1ODk0XzdtAZjmMA0G
    CSqGSIb3DQEBCwUAA4IBAQBsrb/6cxr1LPfwmIK7iBGp9KcLmpKLQy6oPf7i8RoT
    s8iT21TOXf+8iCU8r8rZmDSjYXIg1VAd+9MCahoiduJDZ3p5p1xZ5MTNPfxG7+E0
    NaP/rAw1obG4y2OkNdLE4KRfjR7I9OwvhZH9FDqDUE3zKH2FySy4OXgrH5wQRetC
    ONCNy8IleVJFfb48lDFwG+w88uxA5HLm4q1TijUA5ntiUClnEDf48L+HhROF2bTt
    zQhNSGqvIXwoC/Ep6W21kyZjUIa79s+eGp/bTW1kIsHYX2CfA0yQRVxQQWjsilFy
    UEl1c6RCZPIIbYPHfg2llfpFgoEi7qK1DBA2i/wjhNQ1
    -----END CERTIFICATE-----

Example state:

[root@saltmaster salt]# cat modtest.sls
show ca:
  module.run:
  - file.read:
    - path: /opt/certs/fakeca.crt

ca to mine:
  module.run:
  - mine.send:
    - name: vault_ca_cert
    - mine_function: file.read
    - /opt/certs/fakeca.crt
[root@saltmaster salt]# salt saltmaster state.sls modtest
saltmaster:
----------
          ID: show ca
    Function: module.run
      Result: True
     Comment: file.read: -----BEGIN CERTIFICATE-----
              MIIEbTCCA1WgAwIBAgIYMC4zNzMwMzkxNTg2Nzk1ODk0XzdtAZjmMA0GCSqGSIb3
              DQEBCwUAMHAxCzAJBgNVBAYTAlVTMREwDwYDVQQIDAhNYXJ5bGFuZDESMBAGA1UE
              BwwJQmFsdGltb3JlMQ8wDQYDVQQKDAZTdW5heXUxDzANBgNVBAMMBmZha2VjYTEY
              MBYGCSqGSIb3DQEJARYJbm9Abm8uY29tMB4XDTIwMDQxMzE2MzgxNFoXDTIxMDQx
              MzE2MzgxNFowcDELMAkGA1UEBhMCVVMxETAPBgNVBAgMCE1hcnlsYW5kMRIwEAYD
              VQQHDAlCYWx0aW1vcmUxDzANBgNVBAoMBlN1bmF5dTEPMA0GA1UEAwwGZmFrZWNh
              MRgwFgYJKoZIhvcNAQkBFglub0Buby5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IB
              DwAwggEKAoIBAQDJ/ih/h8NkLWa1hO2MtFyoLoa5woVWm8cZd7iUtSzk+fECurw9
              K9tYudHGBh4jgvftQn6Pk0j2Oqm1efzYFbHK4/tWVGEtV05bVjOPVZdU3QIpKOtk
              ydEaVrzSn+d9TB8uRTFetLKbC/A+tpZr9mJ/CRh4ETCm5y3gYELS/ciHAQt9+Nep
              BRCRBRZzsT7+LRAxwXyDgZwftjK8mtUF12xwtEDd/OYNV2QjBQDKsCafOW27fKeu
              7lpQMiM9WEVilzf3zafEqsNH4bPkMSfYbUWX8c3io2BIzfhT80F9qb747cYts/ko
              FUvqh4jvbOE/Z9WpMX3Dl3hrDtx1542fvExNAgMBAAGjgfowgfcwEgYDVR0TAQH/
              BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAQYwHQYDVR0OBBYEFP/XcjGriiZYLbYH
              genbPqArCvI6MIGxBgNVHSMEgakwgaaAFP/XcjGriiZYLbYHgenbPqArCvI6oXSk
              cjBwMQswCQYDVQQGEwJVUzERMA8GA1UECAwITWFyeWxhbmQxEjAQBgNVBAcMCUJh
              bHRpbW9yZTEPMA0GA1UECgwGU3VuYXl1MQ8wDQYDVQQDDAZmYWtlY2ExGDAWBgkq
              hkiG9w0BCQEWCW5vQG5vLmNvbYIYMC4zNzMwMzkxNTg2Nzk1ODk0XzdtAZjmMA0G
              CSqGSIb3DQEBCwUAA4IBAQBsrb/6cxr1LPfwmIK7iBGp9KcLmpKLQy6oPf7i8RoT
              s8iT21TOXf+8iCU8r8rZmDSjYXIg1VAd+9MCahoiduJDZ3p5p1xZ5MTNPfxG7+E0
              NaP/rAw1obG4y2OkNdLE4KRfjR7I9OwvhZH9FDqDUE3zKH2FySy4OXgrH5wQRetC
              ONCNy8IleVJFfb48lDFwG+w88uxA5HLm4q1TijUA5ntiUClnEDf48L+HhROF2bTt
              zQhNSGqvIXwoC/Ep6W21kyZjUIa79s+eGp/bTW1kIsHYX2CfA0yQRVxQQWjsilFy
              UEl1c6RCZPIIbYPHfg2llfpFgoEi7qK1DBA2i/wjhNQ1
              -----END CERTIFICATE-----
     Started: 16:47:02.592419
    Duration: 8.777 ms
     Changes:
              ----------
              file.read:
                  -----BEGIN CERTIFICATE-----
                  MIIEbTCCA1WgAwIBAgIYMC4zNzMwMzkxNTg2Nzk1ODk0XzdtAZjmMA0GCSqGSIb3
                  DQEBCwUAMHAxCzAJBgNVBAYTAlVTMREwDwYDVQQIDAhNYXJ5bGFuZDESMBAGA1UE
                  BwwJQmFsdGltb3JlMQ8wDQYDVQQKDAZTdW5heXUxDzANBgNVBAMMBmZha2VjYTEY
                  MBYGCSqGSIb3DQEJARYJbm9Abm8uY29tMB4XDTIwMDQxMzE2MzgxNFoXDTIxMDQx
                  MzE2MzgxNFowcDELMAkGA1UEBhMCVVMxETAPBgNVBAgMCE1hcnlsYW5kMRIwEAYD
                  VQQHDAlCYWx0aW1vcmUxDzANBgNVBAoMBlN1bmF5dTEPMA0GA1UEAwwGZmFrZWNh
                  MRgwFgYJKoZIhvcNAQkBFglub0Buby5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IB
                  DwAwggEKAoIBAQDJ/ih/h8NkLWa1hO2MtFyoLoa5woVWm8cZd7iUtSzk+fECurw9
                  K9tYudHGBh4jgvftQn6Pk0j2Oqm1efzYFbHK4/tWVGEtV05bVjOPVZdU3QIpKOtk
                  ydEaVrzSn+d9TB8uRTFetLKbC/A+tpZr9mJ/CRh4ETCm5y3gYELS/ciHAQt9+Nep
                  BRCRBRZzsT7+LRAxwXyDgZwftjK8mtUF12xwtEDd/OYNV2QjBQDKsCafOW27fKeu
                  7lpQMiM9WEVilzf3zafEqsNH4bPkMSfYbUWX8c3io2BIzfhT80F9qb747cYts/ko
                  FUvqh4jvbOE/Z9WpMX3Dl3hrDtx1542fvExNAgMBAAGjgfowgfcwEgYDVR0TAQH/
                  BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAQYwHQYDVR0OBBYEFP/XcjGriiZYLbYH
                  genbPqArCvI6MIGxBgNVHSMEgakwgaaAFP/XcjGriiZYLbYHgenbPqArCvI6oXSk
                  cjBwMQswCQYDVQQGEwJVUzERMA8GA1UECAwITWFyeWxhbmQxEjAQBgNVBAcMCUJh
                  bHRpbW9yZTEPMA0GA1UECgwGU3VuYXl1MQ8wDQYDVQQDDAZmYWtlY2ExGDAWBgkq
                  hkiG9w0BCQEWCW5vQG5vLmNvbYIYMC4zNzMwMzkxNTg2Nzk1ODk0XzdtAZjmMA0G
                  CSqGSIb3DQEBCwUAA4IBAQBsrb/6cxr1LPfwmIK7iBGp9KcLmpKLQy6oPf7i8RoT
                  s8iT21TOXf+8iCU8r8rZmDSjYXIg1VAd+9MCahoiduJDZ3p5p1xZ5MTNPfxG7+E0
                  NaP/rAw1obG4y2OkNdLE4KRfjR7I9OwvhZH9FDqDUE3zKH2FySy4OXgrH5wQRetC
                  ONCNy8IleVJFfb48lDFwG+w88uxA5HLm4q1TijUA5ntiUClnEDf48L+HhROF2bTt
                  zQhNSGqvIXwoC/Ep6W21kyZjUIa79s+eGp/bTW1kIsHYX2CfA0yQRVxQQWjsilFy
                  UEl1c6RCZPIIbYPHfg2llfpFgoEi7qK1DBA2i/wjhNQ1
                  -----END CERTIFICATE-----
----------
          ID: ca to mine
    Function: module.run
      Result: True
     Comment:
     Started: 16:47:02.601397
    Duration: 3.701 ms
     Changes:

Summary for saltmaster
------------
Succeeded: 2 (changed=1)
Failed:    0
------------
Total states run:     2
Total run time:  12.478 ms

Error in minion log:

2020-04-13 16:45:55,445 [salt.loaded.int.module.mine:259 ][ERROR   ][15445] Function file.read in mine.send failed to execute
2020-04-13 16:45:55,446 [salt.loaded.int.module.mine:260 ][DEBUG   ][15445] Error: Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/modules/mine.py", line 255, in send
    **kwargs
  File "/usr/lib/python3.6/site-packages/salt/utils/functools.py", line 142, in call_function
    return salt_function(*function_args, **function_kwargs)
  File "/usr/lib/python3.6/site-packages/salt/modules/file.py", line 3597, in read
    with salt.utils.files.fopen(path, access_mode) as file_obj:
  File "/usr/lib/python3.6/site-packages/salt/utils/files.py", line 399, in fopen
    f_handle = open(*args, **kwargs)  # pylint: disable=resource-leakage
FileNotFoundError: [Errno 2] No such file or directory: 'vault_ca_cert'

2020-04-13 16:45:55,446 [salt.state       :320 ][INFO    ][15445] No changes made for ['mine.send']

Trying to pass in via keyword arg also fails:

[root@saltmaster salt]# cat modtest.sls
ca to mine:
  module.run:
  - mine.send:
    - name: vault_ca_cert
    - mine_function: file.read
    - path: /opt/certs/fakeca.crt
[root@saltmaster salt]# salt saltmaster state.sls modtest
saltmaster:
----------
          ID: ca to mine
    Function: module.run
      Result: True
     Comment:
     Started: 16:48:32.079404
    Duration: 10.471 ms
     Changes:

Summary for saltmaster
------------
Succeeded: 1
Failed:    0
------------
Total states run:     1
Total run time:  10.471 ms
2020-04-13 16:47:48,563 [salt.loaded.int.module.mine:260 ][DEBUG   ][15605] Error: Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/modules/mine.py", line 255, in send
    **kwargs
  File "/usr/lib/python3.6/site-packages/salt/utils/functools.py", line 140, in call_function
    'got only {1}'.format(_exp_prm, _passed_prm))
salt.exceptions.SaltInvocationError: Function expects 1 positional parameters, got only 0

Note this is with the following set in minion config:

log_level_logfile: debug
use_superseded:
- module.run

@garethgreenaway garethgreenaway self-assigned this Apr 13, 2020
@cmcmarrow
Copy link
Contributor

@tazaki Thanks for all the detail.

@cmcmarrow cmcmarrow added v3000.1 vulnerable version Bug broken, incorrect, or confusing behavior labels Apr 13, 2020
@cmcmarrow cmcmarrow added this to the Approved milestone Apr 13, 2020
@garethgreenaway
Copy link
Contributor

@tazaki Thanks for the report. The example of sending along the file path as a keyword argument is failing because in the file.read function it is not a keyword argument, so when the arguments are being checked it is not being seen as a non-keyword argument so the check for the right number of arguments is failing. Still diving in and see what is happening with the secondary scenario.

@sagetherage sagetherage added Regression The issue is a bug that breaks functionality known to work in previous releases. severity-high 2nd top severity, seen by most users, causes major problems ZRelease-Sodium retired label and removed v3000.1 vulnerable version labels Apr 14, 2020
@sagetherage sagetherage added this to In progress in Sodium Apr 14, 2020
@OrangeDog
Copy link
Contributor

there is a bug with passing arguments (non-key word) into the mine.send function

That's #56496

Sodium automation moved this from In progress to Done Apr 18, 2020
@github-abcde
Copy link
Contributor

As the person who introduced piping mine functions through utils.functools, I'll put in my $.02

@tazaki: I expect that this (slight modification of your 2nd attempt) should work:

vault init_secrets mine:
  module.run:
  - mine.send:
    - vault_init_secrets
    - mine_function: vault_helper.get_init_secrets
    - /root/init_secrets.yml

The thing is that mine.send does not have a func argument (anymore, as it has been renamed to name), but it somehow seems to try to execute vault_init_secrets as a module.function (which is not working, because it is an invalid name for a module.function as it does not contain a period)
Alternatively, the first example, using the legacy module.run-format might work (saying this without trying it myself) when providing m_name instead of func (which is what the error-message is trying to hint at, though apparently not clearly enough):

vault init_secrets mine:
  module.run:
  - name: mine.send
  - m_name: vault_init_secrets
  - kwargs:
      mine_function: vault_helper.get_init_secrets
  - args:
    - /root/init_secrets.yml

The m_name argument is one of the reasons for the new module.run-format to exist. This is needed if you want to pass a name-argument to the module-function being run, but not want module.run itself to use it.

@garethgreenaway I'm not entirely in agreement with the change you introduced in #56635 as this deviates from the format used in cmd.run where all arguments, keyword or not, are passed in a single list (i.e., only as args). This is something I was explicitly told to adhere to when discussing my implementation on slack for #55760, which is why utils.mine.parse_function_definition exists.
As cmd.run is now (since 55760) also using utils.functools.call_function to call salt functions, this change in 56635 has a lot more impact than you might have expected.

As for the example

ca to mine:
  module.run:
  - mine.send:
    - name: vault_ca_cert
    - mine_function: file.read
    - path: /opt/certs/fakeca.crt

The kwarg path matches the name of the arg of file.read and thus should be removed from the kwargs and passed as arg to file.read (and not trigger an exception). This is what should be fixed (which is similar to the problem in #56496).

And lastly the example

ca to mine:
  module.run:
  - mine.send:
    - name: vault_ca_cert
    - mine_function: file.read
    - /opt/certs/fakeca.crt

This is a bit of an oddity, as the name-argument appears to be passed to the mine_function instead of being consumed by mine.send. This is a different problem than described above in the previous example, and should be handled as a separate issue.

@tazaki
Copy link
Author

tazaki commented Apr 29, 2020

@github-abcde I'll try this shortly and post the results. Thank you!

@tazaki
Copy link
Author

tazaki commented Apr 29, 2020

Tried it. Different error.

Note I did put this in my minion config:

use_superseded:
  - module.run

Here is the command I ran:

[root@saltmaster ~]# salt minion1 state.sls vault.pki.init_secrets_mine
minion1:
----------
          ID: vault init_secrets mine
    Function: module.run
      Result: True
     Comment:
     Started: 15:00:34.555131
    Duration: 3.121 ms
     Changes:

Summary for minion1
------------
Succeeded: 1
Failed:    0
------------
Total states run:     1
Total run time:   3.121 ms

Here is the state:

vault init_secrets mine:
  module.run:
  - mine.send:
    - vault_init_secrets
    - mine_function: vault_helper.get_init_secrets
    - /root/init_secrets.yml

Here is the module:

import logging
log = logging.getLogger(__name__)
import yaml

def get_init_secrets(init_secrets_path="/opt/vault/init_secrets.yml"):
    with open(init_secrets_path) as stream:
        try:
            return yaml.safe_load(stream)
        except yaml.YAMLError as exc:
            return exc

Here is from the minion debug log:

2020-04-29 15:00:34,552 [salt.template    :120 ][DEBUG   ][8487] Rendered data from file: /var/cache/salt/minion/files/base/vault/pki/init_secrets_mine.sls:
vault init_secrets mine:
  module.run:
  - mine.send:
    - vault_init_secrets
    - mine_function: vault_helper.get_init_secrets
    - /root/init_secrets.yml

2020-04-29 15:00:34,553 [salt.loaded.int.render.yaml:80  ][DEBUG   ][8487] Results of YAML rendering:
OrderedDict([('vault init_secrets mine', OrderedDict([('module.run', [OrderedDict([('mine.send', ['vault_init_secrets', OrderedDict([('mine_function', 'vault_helper.get_init_secrets')]), '/root/init_secrets.yml'])])])]))])
2020-04-29 15:00:34,553 [salt.template    :30  ][PROFILE ][8487] Time (in seconds) to render '/var/cache/salt/minion/files/base/vault/pki/init_secrets_mine.sls' using 'yaml' renderer: 0.0005979537963867188
2020-04-29 15:00:34,554 [salt.utils.lazy  :104 ][DEBUG   ][8487] LazyLoaded config.option
2020-04-29 15:00:34,555 [salt.utils.lazy  :104 ][DEBUG   ][8487] LazyLoaded module.run
2020-04-29 15:00:34,555 [salt.state       :1867][INFO    ][8487] Running state [vault init_secrets mine] at time 15:00:34.555131
2020-04-29 15:00:34,555 [salt.state       :1900][INFO    ][8487] Executing state module.run for [vault init_secrets mine]
2020-04-29 15:00:34,556 [salt.utils.lazy  :104 ][DEBUG   ][8487] LazyLoaded mine.send
2020-04-29 15:00:34,557 [salt.utils.lazy  :104 ][DEBUG   ][8487] LazyLoaded vault_helper.get_init_secrets
2020-04-29 15:00:34,557 [salt.loaded.int.module.mine:259 ][ERROR   ][8487] Function vault_helper.get_init_secrets in mine.send failed to execute
2020-04-29 15:00:34,557 [salt.loaded.int.module.mine:260 ][DEBUG   ][8487] Error: Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/modules/mine.py", line 255, in send
    **kwargs
  File "/usr/lib/python3.6/site-packages/salt/utils/functools.py", line 142, in call_function
    return salt_function(*function_args, **function_kwargs)
TypeError: get_init_secrets() got multiple values for argument 'init_secrets_path'

2020-04-29 15:00:34,558 [salt.state       :320 ][INFO    ][8487] No changes made for ['mine.send']
2020-04-29 15:00:34,558 [salt.state       :2045][INFO    ][8487] Completed state [vault init_secrets mine] at time 15:00:34.558252 (duration_in_ms=3.121)

Edit adding versions:

Salt Version:
           Salt: 3000.1

Dependency Versions:
           cffi: 1.14.0
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.11.1
        libgit2: Not Installed
       M2Crypto: 0.35.2
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.6.2
   mysql-python: Not Installed
      pycparser: 2.20
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.6.8 (default, Apr  2 2020, 13:34:55)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 15.3.0
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.1.4

System Versions:
           dist: centos 7.6.1810 Core
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-957.12.2.el7.x86_64
         system: Linux
        version: CentOS Linux 7.6.1810 Core

@tazaki
Copy link
Author

tazaki commented Apr 29, 2020

I realized I changed my init_secrets_path from an arg to kwarg from testing above. The following worked!

vault init_secrets mine:
  module.run:
  - mine.send:
    - vault_init_secrets
    - mine_function: vault_helper.get_init_secrets
    - init_secrets_path: /root/init_secrets.yml

Note changing the kwarg to arg also works with your suggestion:

vault init_secrets mine:
  module.run:
  - mine.send:
    - vault_init_secrets
    - mine_function: vault_helper.get_init_secrets
    - /root/init_secrets.yml
import logging
log = logging.getLogger(__name__)
import yaml

def get_init_secrets(init_secrets_path):
    with open(init_secrets_path) as stream:
        try:
            return yaml.safe_load(stream)
        except yaml.YAMLError as exc:
            return exc

@tazaki
Copy link
Author

tazaki commented Apr 29, 2020

Thank you so much @github-abcde !

TeddyAndrieux added a commit to scality/metalk8s that referenced this issue Apr 30, 2020
In Salt '3000.2' we can't provide kwargs for mine call, so we need to
use potitional arguments in pillar and for `module.run` with `mine.send`
we need to use potitional arguments to provide the mine name, as `func`
get renamed to `name` but this arguments is gave to the function behind,
which make the mine function call fail.

Sees saltstack/salt#56584
TeddyAndrieux added a commit to scality/metalk8s that referenced this issue Apr 30, 2020
In Salt '3000.2' we can't provide kwargs for mine call, so we need to
use potitional arguments in pillar and for `module.run` with `mine.send`
we need to use potitional arguments to provide the mine name, as `func`
get renamed to `name` but this arguments is gave to the function behind,
which make the mine function call fail.

Sees saltstack/salt#56584
hatifnatt added a commit to hatifnatt/pki-formula that referenced this issue Dec 3, 2020
Add compatibility for new `module.run` call style
Add compatibility with Salt >= 3000 where mine functions changed a lot
 saltstack/salt@da29e15#diff-a64f3cb030f6172ac00301df45b9a678e36cbb2cac6d16c62e31cd8adfb1334cL187-R204
 saltstack/salt#56584 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug broken, incorrect, or confusing behavior Regression The issue is a bug that breaks functionality known to work in previous releases. severity-high 2nd top severity, seen by most users, causes major problems ZRelease-Sodium retired label
Projects
No open projects
Sodium
  
Done
6 participants