Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2016.3.0 FreeBSD Failed to load grains defined in grain file disks.disks in function <function disks at 0x80cff9320>, error: #33578

Closed
ohauer opened this issue May 27, 2016 · 14 comments
Labels
Bug broken, incorrect, or confusing behavior cannot-reproduce cannot be replicated with info/context provided P2 Priority 2 Platform Relates to OS, containers, platform-based utilities like FS, system based apps severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around
Milestone

Comments

@ohauer
Copy link

ohauer commented May 27, 2016

Description of Issue/Question

The following error message is spamming my master log


[salt.loader                                                ][CRITICAL][80377] Failed to load grains defined in grain file disks.disks in function <function disks at 0x80cff9140>, error:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/salt/loader.py", line 724, in grains
    ret = fun()
  File "/usr/local/lib/python2.7/site-packages/salt/grains/disks.py", line 31, in disks
    return _freebsd_geom()
  File "/usr/local/lib/python2.7/site-packages/salt/grains/disks.py", line 107, in _freebsd_geom
    parse_geom_attribs(device)
  File "/usr/local/lib/python2.7/site-packages/salt/grains/disks.py", line 94, in parse_geom_attribs
    search.group(1))
  File "/usr/local/lib/python2.7/site-packages/salt/grains/disks.py", line 69, in _datavalue
    return int(data)
ValueError: invalid literal for int() with base 10: 'unknown'

Setup

happens existing and/or on fresh master without SLS files

Steps to Reproduce Issue

run salt-master on FreeBSD

Versions Report

salt --versions-report
Salt Version:
Salt: 2016.3.0

Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.5.0
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.8
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.7
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pygit2: Not Installed
Python: 2.7.11 (default, May 9 2016, 20:02:36)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 15.2.0
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.3
ZMQ: 4.1.4

System Versions:
dist:
machine: amd64
release: 10.3-RELEASE-p3
system: FreeBSD
version: Not Installed

@Ch3LL
Copy link
Contributor

Ch3LL commented May 27, 2016

@ohauer I am not seeing these errors in these logs. Did you upgrade from a previous version? How did you install salt? Any other information that might help to replicate?

@Ch3LL Ch3LL added the info-needed waiting for more info label May 27, 2016
@Ch3LL Ch3LL added this to the Blocked milestone May 27, 2016
@Ch3LL Ch3LL added the cannot-reproduce cannot be replicated with info/context provided label May 27, 2016
@Whoops
Copy link
Contributor

Whoops commented May 28, 2016

I am also seeing this issue. I did upgrade salt from 2015.8.8_1 -> 2016.3.0. Salt was installed via the sysutils/py-salt port.

My minion logs also display this when running state.highstate.

I'm not sure if it's helpful, but when running grains.items there is no disk key. Also this system was installed and boots directly from zfs. I could be wrong, but I don't think geom is involved, so the problem might be related to parse_geom_attribs

@kev009
Copy link
Contributor

kev009 commented May 28, 2016

Can you paste the output of 'geom disk list'?

@ohauer
Copy link
Author

ohauer commented May 28, 2016

Sure, that's from one of my masters/syndic systems

geom disk list

Geom name: cd0
Providers:

  1. Name: cd0
    Mediasize: 0 (0B)
    Sectorsize: 2048
    Mode: r0w0e0
    descr: NECVMWar VMware IDE CDR10
    ident: (null)
    rotationrate: unknown
    fwsectors: 0
    fwheads: 0

Geom name: da0
Providers:

  1. Name: da0
    Mediasize: 8589934592 (8.0G)
    Sectorsize: 512
    Mode: r5w5e14
    descr: VMware Virtual disk
    ident: (null)
    rotationrate: unknown
    fwsectors: 63
    fwheads: 255

geom disk list (on a bare metal master in a remote office)

Geom name: cd0
Providers:

  1. Name: cd0
    Mediasize: 0 (0B)
    Sectorsize: 2048
    Mode: r0w0e0
    descr: JLMS DVD-ROM LTD-166S
    ident: (null)
    rotationrate: unknown
    fwsectors: 0
    fwheads: 0

Geom name: ada0
Providers:

  1. Name: ada0
    Mediasize: 128035676160 (119G)
    Sectorsize: 512
    Stripesize: 4096
    Stripeoffset: 0
    Mode: r5w5e14
    descr: Samsung SSD 840 PRO Series
    lunid: 50025385a01e580c
    ident: S1ANNSAF241659N
    rotationrate: 0
    fwsectors: 63
    fwheads: 16

Geom name: ada1
Providers:

  1. Name: ada1
    Mediasize: 2000398934016 (1.8T)
    Sectorsize: 512
    Stripesize: 4096
    Stripeoffset: 0
    Mode: r1w1e1
    descr: WDC WD2003FZEX-00Z4SA0
    lunid: 50014ee004070466
    ident: WD-WMC1P0F03969
    rotationrate: 7200
    fwsectors: 63
    fwheads: 16

Geom name: ada2
Providers:

  1. Name: ada2
    Mediasize: 2000398934016 (1.8T)
    Sectorsize: 512
    Stripesize: 4096
    Stripeoffset: 0
    Mode: r1w1e1
    descr: WDC WD2002FAEX-007BA0
    lunid: 50014ee65633b408
    ident: WD-WMAY01073054
    rotationrate: unknown
    fwsectors: 63
    fwheads: 16

@ohauer
Copy link
Author

ohauer commented May 28, 2016

If uploaded an archive with the output of grains.items from the same system
Perhaps it helps to track the issue down.
http://people.freebsd.org/~ohauer/diffs/PR33578_saltstack.tar.gz

Archive contains output on FreeBSD 10.x with salt 2015.5.8, 2015.8.10 and 2016.3.0

@AlexanderThaller
Copy link
Contributor

AlexanderThaller commented May 28, 2016

I'm having the same error when running salt-call pillar.items (or any other command) on FreeBSD with a FreeBSD master (they are both on the same machine):

[CRITICAL] Failed to load grains defined in grain file disks.disks in function <function disks at 0x80bfaaed8>, error:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/salt/loader.py", line 724, in grains
    ret = fun()
  File "/usr/local/lib/python2.7/site-packages/salt/grains/disks.py", line 31, in disks
    return _freebsd_geom()
  File "/usr/local/lib/python2.7/site-packages/salt/grains/disks.py", line 107, in _freebsd_geom
    parse_geom_attribs(device)
  File "/usr/local/lib/python2.7/site-packages/salt/grains/disks.py", line 94, in parse_geom_attribs
    search.group(1))
  File "/usr/local/lib/python2.7/site-packages/salt/grains/disks.py", line 69, in _datavalue
    return int(data)
ValueError: invalid literal for int() with base 10: 'unknown'
local:
    ----------

Versions Report

Master

Salt Version:
           Salt: 2016.3.0

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.5.0
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.7
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.11 (default, May 19 2016, 01:18:07)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.3
            ZMQ: 4.1.4

System Versions:
           dist:   
        machine: amd64
        release: 10.3-RELEASE-p3
         system: FreeBSD
        version: Not Installed

Minion

Salt Version:
           Salt: 2016.3.0

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.5.0
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.7
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.11 (default, May 19 2016, 01:18:07)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.3
            ZMQ: 4.1.4

System Versions:
           dist:   
        machine: amd64
        release: 10.3-RELEASE-p3
         system: FreeBSD
        version: Not Installed

I installed via pkng and the latest FreeBSD pkg repo.

Geom Disk List

Geom name: ada0
Providers:
1. Name: ada0
   Mediasize: 1073741824000 (1.0T)
   Sectorsize: 512
   Mode: r2w2e3
   descr: VBOX HARDDISK
   ident: VBad682683-12cb3f7d
   rotationrate: unknown
   fwsectors: 63
   fwheads: 16

Geom name: cd0
Providers:
1. Name: cd0
   Mediasize: 244047872 (233M)
   Sectorsize: 2048
   Mode: r0w0e0
   descr: VBOX CD-ROM
   ident: (null)
   rotationrate: unknown
   fwsectors: 0
   fwheads: 0

@AlexanderThaller
Copy link
Contributor

Works fine with older version:

[INFO    ] Determining pillar cache
[INFO    ] Determining pillar cache
local:
    ----------

Versions Report

Master

Salt Version:
           Salt: 2015.8.8

Dependency Versions:
         Jinja2: 2.8
       M2Crypto: Not Installed
           Mako: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
         Python: 2.7.11 (default, May 19 2016, 01:23:53)
           RAET: Not Installed
        Tornado: 4.3
            ZMQ: 4.1.4
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.5.0
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
        libgit2: Not Installed
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.7
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: Not Installed
          smmap: Not Installed
        timelib: Not Installed

System Versions:
           dist:   
        machine: amd64
        release: 10.3-RELEASE-p3

Minion

Salt Version:
           Salt: 2015.8.8

Dependency Versions:
         Jinja2: 2.8
       M2Crypto: Not Installed
           Mako: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
         Python: 2.7.11 (default, May 19 2016, 01:23:53)
           RAET: Not Installed
        Tornado: 4.3
            ZMQ: 4.1.4
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.5.0
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
        libgit2: Not Installed
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.7
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: Not Installed
          smmap: Not Installed
        timelib: Not Installed

System Versions:
           dist:   
        machine: amd64
        release: 10.3-RELEASE-p3

@kev009
Copy link
Contributor

kev009 commented May 28, 2016

Ok, sorry about this :( it seems there are scenarios where rotationrate will be (unknown) but I was trying to cast it to an Integer to make usage easier elsewhere.

#33604 has a fix to make the data parsing more resilient. @cedwards, could you pull this patch into the port?

I polled twitter for more geom disk list samples so will try and write some unit test next.

@ohauer
Copy link
Author

ohauer commented May 29, 2016

@kev009
after applying #33604 the issue is no longer present

cachedout pushed a commit that referenced this issue May 31, 2016
@cachedout
Copy link
Contributor

Closed via #33604

@Ch3LL Ch3LL added Bug broken, incorrect, or confusing behavior P2 Priority 2 Platform Relates to OS, containers, platform-based utilities like FS, system based apps severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around and removed info-needed waiting for more info labels May 31, 2016
@amontalban
Copy link
Contributor

Hey @kev009,

After applying this patch I'm still seeing this in a VirtualBox VM:

root@:~ # uname -a
FreeBSD  10.2-RELEASE-p18 FreeBSD 10.2-RELEASE-p18 #0: Sat May 28 08:53:43 UTC 2016     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64
root@:~ # salt-call --local pkg.latest_version bash
[WARNING ] Failed to open log file, do you have permission to write to /var/log/salt/minion?
[CRITICAL] Failed to load grains defined in grain file disks.disks in function <function disks at 0x80bf9be60>, error:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/salt/loader.py", line 724, in grains
    ret = fun()
  File "/usr/local/lib/python2.7/site-packages/salt/grains/disks.py", line 31, in disks
    return _freebsd_geom()
  File "/usr/local/lib/python2.7/site-packages/salt/grains/disks.py", line 118, in _freebsd_geom
    parse_geom_attribs(device)
  File "/usr/local/lib/python2.7/site-packages/salt/grains/disks.py", line 113, in parse_geom_attribs
    if tmp[_geomconsts.ROTATIONRATE] == 0:
KeyError: 'rotationrate'
[INFO    ] Executing command ['pkg', 'info', '-ao'] in directory '/root'
[INFO    ] Executing command ['pkg', '--version'] in directory '/root'
[INFO    ] Executing command ['pkg', 'search', '-S', 'name', '-Q', 'version', '-e', '-q', 'bash'] in directory '/root'
local:
    4.3.42_1
root@:~ # geom disk list
Geom name: ada0
Providers:
1. Name: ada0
   Mediasize: 22549626880 (21G)
   Sectorsize: 512
   Mode: r2w2e5
   descr: VBOX HARDDISK
   ident: VBe248e9aa-bbc8d3a1
   fwsectors: 1
   fwheads: 1

In all boxes that I have access I do not see the "rotationrate" keyword in the geom disk list.

Let me know if you need some testing.

Thanks!

@amontalban
Copy link
Contributor

@kev009 / @cachedout Not sure if #33767 sounds good enough but it's not failing anymore for me.

Thanks!

@kev009
Copy link
Contributor

kev009 commented Jun 4, 2016

rotationrate will only be there on 10.3 and 11.0 and #33767 is the right intent.

Sorry for all the fail around this grain 😞

@amontalban
Copy link
Contributor

Fixed #33767 based on @kev009 recommendation to cover FreeBSD < 10.3.

Thanks!

uqs pushed a commit to freebsd/freebsd-ports that referenced this issue Jun 15, 2016
The patch fixes some issues that are new with 2016.3.0
- saltstack/salt#33529
- saltstack/salt#33554
- saltstack/salt#33578
- saltstack/salt#33608

PR:		209829
Submitted by:	ohauer, Andreas Montalban <amontalban@gmail.com>
Approved by:	christer.edwards@gmail.com (maintainer timeout)


git-svn-id: svn+ssh://svn.freebsd.org/ports/head@416914 35697150-7ecd-e111-bb59-0022644237b5
uqs pushed a commit to freebsd/freebsd-ports that referenced this issue Jun 15, 2016
The patch fixes some issues that are new with 2016.3.0
- saltstack/salt#33529
- saltstack/salt#33554
- saltstack/salt#33578
- saltstack/salt#33608

PR:		209829
Submitted by:	ohauer, Andreas Montalban <amontalban@gmail.com>
Approved by:	christer.edwards@gmail.com (maintainer timeout)
svmhdvn pushed a commit to svmhdvn/freebsd-ports that referenced this issue Jan 10, 2024
The patch fixes some issues that are new with 2016.3.0
- saltstack/salt#33529
- saltstack/salt#33554
- saltstack/salt#33578
- saltstack/salt#33608

PR:		209829
Submitted by:	ohauer, Andreas Montalban <amontalban@gmail.com>
Approved by:	christer.edwards@gmail.com (maintainer timeout)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug broken, incorrect, or confusing behavior cannot-reproduce cannot be replicated with info/context provided P2 Priority 2 Platform Relates to OS, containers, platform-based utilities like FS, system based apps severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around
Projects
None yet
Development

No branches or pull requests

7 participants