Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Salt CLI is rounding floats to 2 decimal places #45620

Open
jmarinaro opened this issue Jan 22, 2018 · 7 comments

Comments

Projects
None yet
3 participants
@jmarinaro
Copy link
Contributor

commented Jan 22, 2018

Description of Issue/Question

When passing arguments via the command line, salt is rounding floats to 2 decimal places. I have verified this behavior exists in develop, 2017.7, and 2016.11. If I pass an argument I expect it to remain as is.

Steps to Reproduce Issue

root@a0e930608610:/# salt-call test.arg_clean dec2='1516645352.09' dec3='1516645352.009' dec4='1516645352.0009'
local:
    ----------
    args:
    kwargs:
        ----------
        dec2:
            1516645352.09
        dec3:
            1516645352.01
        dec4:
            1516645352.0

Versions Report

root@a0e930608610:/# salt-call --versions-report
Salt Version:
           Salt: 2017.7.2

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.4.2
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.12 (default, Dec  4 2017, 14:50:18)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: Ubuntu 16.04 xenial
         locale: ANSI_X3.4-1968
        machine: x86_64
        release: 4.10.0-42-generic
         system: Linux
        version: Ubuntu 16.04 xenial
@erwindon

This comment has been minimized.

Copy link
Contributor

commented Aug 17, 2018

Is the issue really about the number of digits after the decimal ".", or is it more about the total number of digits allowed?

@erwindon

This comment has been minimized.

Copy link
Contributor

commented Aug 17, 2018

Was something changed in 2018.3?
Or is this just different when using Python3, like I do?
In my 2018.3 the result is consistent with a double floating point number as expected:

# salt-call test.arg_clean dec2='1516645352.09' dec3='1516645352.009' dec4='1516645352.0009' dec5='1516645352.00009' dec6='1516645352.000009' dec7='1516645352.0000009'
local:
    ----------
    args:
    kwargs:
        ----------
        dec2:
            1516645352.09
        dec3:
            1516645352.009
        dec4:
            1516645352.0009
        dec5:
            1516645352.00009
        dec6:
            1516645352.000009
        dec7:
            1516645352.000001

That is 16 digits of precision, the typical maximum for IEEE double numbers.

@jmarinaro

This comment has been minimized.

Copy link
Contributor Author

commented Aug 17, 2018

Looks like it must be something with python2.7, I still see the issue with 2018.3.2

root@74ba8eb94a45:/# salt-call --versions-report
Salt Version:
           Salt: 2018.3.2
 
Dependency Versions:
           cffi: 1.5.2
       cherrypy: Not Installed
       dateutil: 2.4.2
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: 0.24.0
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.24.0
         Python: 2.7.12 (default, Dec  4 2017, 14:50:18)
   python-gnupg: 0.3.8
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4
 
System Versions:
           dist: Ubuntu 16.04 xenial
         locale: ANSI_X3.4-1968
        machine: x86_64
        release: 4.15.0-32-generic
         system: Linux
        version: Ubuntu 16.04 xenial
 
root@74ba8eb94a45:/# salt-call test.arg_clean dec2='1516645352.09' dec3='1516645352.009' dec4='1516645352.0009'
local:
    ----------
    args:
    kwargs:
        ----------
        dec2:
            1516645352.09
        dec3:
            1516645352.01
        dec4:
            1516645352.0
@erwindon

This comment has been minimized.

Copy link
Contributor

commented Aug 18, 2018

Did some testing with a pair of 2018.3 systems:
on the python2 based system the original observation can be repeated.
on the python3 based system the original observation cannot be repeated, numbers be have like 64-bit double.

@erwindon

This comment has been minimized.

Copy link
Contributor

commented Aug 18, 2018

And another test done.
The effect only appears with salt-call; not when salt is used to target the same minion.

@erwindon

This comment has been minimized.

Copy link
Contributor

commented Aug 18, 2018

And another test done:
sudo salt-call --out=yaml test.arg_clean dec2='1516645352.09' dec3='1516645352.009' dec4='1516645352.0009'
properly shows the given values in all their details (in yaml)
the default outputter is nested; so that one most likely has the problem...

@erwindon

This comment has been minimized.

Copy link
Contributor

commented Aug 18, 2018

The problem is due to the default format that python2 uses for formatting floating point numbers.
It is actually a old problem, see https://mail.python.org/pipermail/tutor/2007-December/058964.html.
In the Python2 source code it says #define PyFloat_STR_PRECISION 12. Thus the restriction to 12 digits there.
In python issue https://bugs.python.org/issue9337 it was decided that this artificial limitation should be removed. Therefore starting with Python3.2 it behaves differently.
A python function to format to a result according to its true value is available, it is called repr().
A small change to the Salt code lets it use that function. I'll raise a PR for that.

erwindon added a commit to erwindon/salt that referenced this issue Aug 18, 2018

Fix for saltstack#45620: "Salt CLI is rounding floats to 2 decimal pl…
…aces" (actually: Salt CLI is using only 12 digits for precision)

use repr() to get the full precision also for older python versions
as until about python32 it was limited to 12 digits only by default

erwindon added a commit to erwindon/salt that referenced this issue Aug 19, 2018

Fix for saltstack#45620: "Salt CLI is rounding floats to 2 decimal pl…
…aces" (actually: Salt CLI is using only 12 digits for precision)

use repr() to get the full precision also for older python versions
as until about python32 it was limited to 12 digits only by default

gtmanfred added a commit that referenced this issue Aug 20, 2018

Merge pull request #49187 from erwindon/py27fixnumberformat
Fix for #45620: "Salt CLI is rounding floats to 2 decimal places"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.