salt-run jobs.list_jobs data incomplete

[UtahDave]

In the latest 2014.7 branch some of the data is not making it into the job cache:

I'm seeing this on Ubuntu Precise 12.04

vagrant@saltmaster:~$ sudo salt-run jobs.list_jobs
20150310162647657398:
    ----------
    Arguments:
    Function:
        test.ping
    StartTime:
        2015, Mar 10 16:26:47.657398
    Target:
        unknown-target
    Target-type:
    User:
        root
20150310162652669514:
    ----------
    Arguments:
    Function:
        saltutil.find_job
    StartTime:
        2015, Mar 10 16:26:52.669514
    Target:
        unknown-target
    Target-type:
    User:
        root
20150310162837107945:
    ----------
    Arguments:
    Function:
        test.versions_report
    StartTime:
        2015, Mar 10 16:28:37.107945
    Target:
        unknown-target
    Target-type:
    User:
        root

Versions report:

minion1:
               Salt: 2014.7.2-282-gf6f6afe
             Python: 2.7.3 (default, Feb 27 2014, 19:58:35)
             Jinja2: 2.6
           M2Crypto: 0.21.1
     msgpack-python: 0.1.10
       msgpack-pure: Not Installed
           pycrypto: 2.4.1
            libnacl: Not Installed
             PyYAML: 3.10
              ioflo: Not Installed
              PyZMQ: 13.0.0
               RAET: Not Installed
                ZMQ: 3.2.2
               Mako: Not Installed

master versions_report:

vagrant@saltmaster:~$ sudo salt-master --versions-report
           Salt: 2014.7.2-282-gf6f6afe
         Python: 2.7.3 (default, Feb 27 2014, 19:58:35)
         Jinja2: 2.6
       M2Crypto: 0.21.1
 msgpack-python: 0.1.10
   msgpack-pure: Not Installed
       pycrypto: 2.4.1
        libnacl: Not Installed
         PyYAML: 3.10
          ioflo: Not Installed
          PyZMQ: 13.0.0
           RAET: Not Installed
            ZMQ: 3.2.2
           Mako: Not Installed

[jfindlay]

Thanks for contributing to the salt project, @UtahDave!

[terminalmage]

I've submitted a fix, but it may not be 100% complete. Details in this comment: #21723 (comment)

[terminalmage]

Actually, it looks like Dave's fix in #21630 takes care of this.

[ssgward]

@jfindlay - will you validate this fix on 7.3 please.

[jfindlay]

@ssgward, it seems that the pull request reference by @terminalmage has not been merged yet.

[terminalmage]

Yes, that is because I closed it.

[jfindlay]

I think this means this is fixed. I don't see any undefined fields in the jobs data. The jobs shown are for a master and minion process running on the same VM.

centos-7-main salt 2014.7 # salt centos-7-main cmd.run ls /
centos-7-main:
    bin
    boot
    dev
    etc
    home
    lib
    lib64
    media
    mnt
    opt
    proc
    root
    run
    salt-syndic.log
    sbin
    srv
    sys
    tmp
    usr
    var
centos-7-main salt 2014.7 # salt-run jobs.list_jobs
20150319095229207420:
    ----------
    Arguments:
        - ls
        - /
    Function:
        cmd.run
    StartTime:
        2015, Mar 19 09:52:29.207420
    Target:
        centos-7-main
    Target-type:
        glob
    User:
        root

[rallytime]

After reviewing what @jfindlay did and talking with @thatch45, this looks fixed to me.

[Lothiraldan]

Is the fix included in 2015.5.X versions?

[Lothiraldan]

I still have the issue with 2015.5.2 version:

root@vagrant-ubuntu-trusty-64:/home/vagrant/saltpad# sudo rm -Rf /var/cache/salt/master/jobs/
root@vagrant-ubuntu-trusty-64:/home/vagrant/saltpad# sudo restart salt-master
salt-master start/running, process 29818
root@vagrant-ubuntu-trusty-64:/home/vagrant/saltpad# salt -v '*' cmd.run 'ls' cwd='/tmp'
Executing job with jid 20150624151516957386
-------------------------------------------

local.local:
    bootstrap-salt.log
    bootstrap_salt.sh
    minion-seed-keys
    saltpad.log
    tmpDrH1Q3
    tmpfRipxo
    tmpoUMxO6
    tmptkrI3U
    vagrant-shell
root@vagrant-ubuntu-trusty-64:/home/vagrant/saltpad# salt-run jobs.list_jobs20150624151516957386:
    ----------
    Arguments:
    Function:
        cmd.run
    StartTime:
        2015, Jun 24 15:15:16.957386
    Target:
        unknown-target
    Target-type:
    User:
        root

Both minion and master are on the same VM:

root@vagrant-ubuntu-trusty-64:/home/vagrant/saltpad# sudo salt-minion --versions-report
           Salt: 2015.5.2
         Python: 2.7.6 (default, Mar 22 2014, 22:59:56)
         Jinja2: 2.7.3
       M2Crypto: 0.21.1
 msgpack-python: 0.4.6
   msgpack-pure: Not Installed
       pycrypto: 2.6.1
        libnacl: Not Installed
         PyYAML: 3.11
          ioflo: Not Installed
          PyZMQ: 14.0.1
           RAET: Not Installed
            ZMQ: 4.0.4
           Mako: 0.9.1

root@vagrant-ubuntu-trusty-64:/home/vagrant/saltpad# salt-master --versions-report
           Salt: 2015.5.2
         Python: 2.7.6 (default, Mar 22 2014, 22:59:56)
         Jinja2: 2.7.3
       M2Crypto: 0.21.1
 msgpack-python: 0.4.6
   msgpack-pure: Not Installed
       pycrypto: 2.6.1
        libnacl: Not Installed
         PyYAML: 3.11
          ioflo: Not Installed
          PyZMQ: 14.0.1
           RAET: Not Installed
            ZMQ: 4.0.4
           Mako: 0.9.1

I'm using default salt-master config, no external master_job_cache is defined. I'm not using syndic either.

It really looks like the same issue for me, but do you want me to open a new one?

[rallytime]

No, we can reopen this one. Thanks for the extra info.

[cachedout]

I have reproduced this issue at the 2015.5.2 tag as @Lothiraldan mentions, but I can also confirm that this is fixed at the head of the 2015.5 branch. Re-closing.