file.managed state should not download a file if the checksum did not change

[astehlik]

When using the file.managed state with a HTTP URL and a provided source_hash I would expect Salt to check the hash of an existing local file before downloading the file from the server.

This seems like a regression to me because in older versions (just tested with salt-call 0.17.5) the file is only downloaded once. After that, the log just prints File ... is in the correct state

Setup

Salt Version: 2016.3.0 (Boron) - in masterless local mode
Ubuntu 14.04 trusty
Salt config / log output: https://gist.github.com/astehlik/adb8e284cd8974cba0bd2858c5bb038b

Steps to Reproduce Issue

• Create a salt sate with file.managed (see Gist)
• run salt-call state.highstate
• file is downloaded and checksum matches
• run salt-call state.highstate again, file is downloaded again

Versions Report

Working version

$ salt-call --versions-report

           Salt: 0.17.5
         Python: 2.7.6 (default, Jun 22 2015, 17:58:13)
         Jinja2: 2.7.2
       M2Crypto: 0.21.1
 msgpack-python: 0.3.0
   msgpack-pure: Not Installed
       pycrypto: 2.6.1
         PyYAML: 3.10
          PyZMQ: 14.0.1
            ZMQ: 4.0.4

Broken version

$ salt-call --versions-report

Salt Version:
           Salt: 2016.3.0

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 1.5
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 0.9.1
   msgpack-pure: Not Installed
 msgpack-python: 0.3.0
   mysql-python: 1.2.3
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.6 (default, Jun 22 2015, 17:58:13)
   python-gnupg: Not Installed
         PyYAML: 3.10
          PyZMQ: 14.0.1
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.4

System Versions:
           dist: Ubuntu 14.04 trusty
        machine: x86_64
        release: 4.2.0-35-generic
         system: Linux
        version: Ubuntu 14.04 trusty

[timcharper]

I have also begun to notice this, and was just about to submit the same issue.

My versions-report:

Salt Version:
           Salt: 2016.3.0

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: 0.21.1
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.7
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.5 (default, Nov 20 2015, 02:00:19)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.7.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5

System Versions:
           dist: centos 7.2.1511 Core
        machine: x86_64
        release: 3.10.0-327.el7.x86_64
         system: Linux
        version: CentOS Linux 7.2.1511 Core

[timcharper]

It appears that this behavior is also present for http (non-SSL).

[timcharper]

This behavior was working correctly in 2015.8.8.2, but is wrong in 2016.3.0.

[timcharper]

The behavior is also correct in 2015.8.10

[timcharper]

Behavior is also correct in v2016.3.0rc0, but broken in v2016.3.0rc2. I was unable to test v2016.3.0rc1.

So regression appears to have been introduced between v2016.3.0rc0 and v2016.3.0rc2

[Ch3LL]

I am able to replicate this behavior. @timcharper thanks for the investigation work it made for much speedier tests.

Looks like the break was caused by this PR: #31265

Even when adding the new argument - skip_verify: True to the sls it is still downloading the file

this issue also might be related: #33575

ping @terminalmage

[terminalmage]

The fix was made in the 2015.8 release branch and merged forward as we typically do for bugfixes. The merge forward may not have fully propagated this fix, however. I'll look at this on Thursday.

[terminalmage]

I've opened #34010 to fix this.

[ghost]

This bug is still in 2017.7.0 version!

[terminalmage]

@ata-sql Since this issue has been closed for over a year, please file a new issue with the relevant information and version report.