cp.push saved file crashed

[GeniusWiki]

Command looks this:
salt 'db-01' cp.push /var/backups/some.tar.gz

The file downloaded has same size, but the md4sum is different. And gunzip cannot unzip - the original file can be unzipped.

I test same command in other server but different gzip files, looks fine.
I also test this command in same server but a small text file, looks fine as well...

[GeniusWiki]

did further test, move same tar.gz to another server (which is success on another file cp.push), cp.push failed again.

[UtahDave]

So does this only affect gzipped files?

[GeniusWiki]

No , I test push the unzipped file, a plain SQL dump file, still crashed.

On 03/08/2013, at 3:35 AM, David Boucha notifications@github.com wrote:

So does this only affect gzipped files?

—
Reply to this email directly or view it on GitHub.

[terminalmage]

When you say "crashed", what do you mean by this? Is there a traceback? If so, please post it.

Also, please post the output of salt --versions-report on the minion.

[terminalmage]

I've confirmed this bug in develop

The original file:

erik@virtubuntu:~/Downloads% ls -l salt-0.16.0.tar.gz
-rw-r--r-- 1 erik erik 3523789 Aug  8 14:07 salt-0.16.0.tar.gz
erik@virtubuntu:~/Downloads% md5sum salt-0.16.0.tar.gz
4e4f382614b8c30818be0c2ebe2988e8  salt-0.16.0.tar.gz
erik@virtubuntu:~/Downloads% file salt-0.16.0.tar.gz
salt-0.16.0.tar.gz: gzip compressed data, was "dist/salt-0.16.0.tar", last modified: Mon Jul  1 17:53:03 2013, max compression

The file after being pushed to the master:

root@virtubuntu:/home/erik/saltdev/var/cache/salt/master/minions/virtubuntu/files/home/erik/Downloads# ls -l salt-0.16.0.tar.gz
-rw-r--r-- 1 root root 3523789 Aug  8 14:30 salt-0.16.0.tar.gz
root@virtubuntu:/home/erik/saltdev/var/cache/salt/master/minions/virtubuntu/files/home/erik/Downloads# md5sum salt-0.16.0.tar.gz
5cc431689f26a8eaa0bb59a9d620996f  salt-0.16.0.tar.gz
root@virtubuntu:/home/erik/saltdev/var/cache/salt/master/minions/virtubuntu/files/home/erik/Downloads# file salt-0.16.0.tar.gz
salt-0.16.0.tar.gz: data
root@virtubuntu:/home/erik/saltdev/var/cache/salt/master/minions/virtubuntu/files/home/erik/Downloads# tar tvf salt-0.16.0.tar.gz

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

[terminalmage]

Another problem I noticed here is that, if the recv_files file_recv option is not set to True, then cp.push still returns True even though no file was copied.

[terminalmage]

Additionally, it seems that when file_recv is not set, the minion still sends the file up to the master. Here's debug output from the master:

[INFO    ] Clear payload received with command publish
[INFO    ] User root Published command cp.push with jid 20130808163412295980
[DEBUG   ] Published command details {'tgt_type': 'glob', 'jid': '20130808163412295980', 'tgt': 'virtubuntu', 'ret': '', 'user': 'root', 'arg': ['/home/erik/Downloads/salt-0.16.0.tar.gz'], 'fun': 'cp.push'}
[INFO    ] Clear payload received with command _auth
[DEBUG   ] This salt-master instance has accepted 1 minion keys.
[INFO    ] Authentication request from virtubuntu
[INFO    ] Authentication accepted from virtubuntu
[INFO    ] AES payload received with command _file_recv
[INFO    ] AES payload received with command _file_recv
[INFO    ] AES payload received with command _file_recv
[INFO    ] AES payload received with command _file_recv
[INFO    ] AES payload received with command _file_recv
[INFO    ] AES payload received with command _file_recv
[INFO    ] AES payload received with command _file_recv
[INFO    ] AES payload received with command _file_recv
[INFO    ] AES payload received with command _file_recv
[INFO    ] AES payload received with command _file_recv
[INFO    ] AES payload received with command _file_recv
[INFO    ] AES payload received with command _file_recv
[INFO    ] AES payload received with command _file_recv
[INFO    ] AES payload received with command _file_recv
[INFO    ] AES payload received with command _return
[INFO    ] Got return from virtubuntu for job 20130808163412295980

[GeniusWiki]

Looks you reproduced the bug, anyway, put my version report and hope it help.
Salt: 0.16.0
Python: 2.7.3 (default, Apr 10 2013, 06:20:15)
Jinja2: 2.6
M2Crypto: 0.21.1
msgpack-python: 0.1.10
msgpack-pure: Not Installed
pycrypto: 2.4.1
PyYAML: 3.10
PyZMQ: 13.0.0
ZMQ: 3.2.2

As I used cp.push in 2 different server for daily backup, looks the problem related to file size. The smaller one always success. The large one >10M fail always.

[terminalmage]

@GeniusWiki Thanks! But can you clarify further what you mean by the cp.push "crashing"?

[GeniusWiki]

Just md5sum checking failed and gunzip failed, same as your test...

[terminalmage]

OK. I was just concerned that there were tracebacks I wasn't catching. I found a traceback when no file_recv option is set in the master config file, which I have fixed.

Thanks for the report, we'll look into this.

[GeniusWiki]

I confirmed my configuration:
file_recv: True

[terminalmage]

Yeah, the issue that caused the traceback I found earlier is that there was no default value when file_recv isn't included in the master config file.

[anitakrueger]

Wondering what the word is on this one?

I tried the same thing:
salt-call cp.push /tmp/base.tar.gz
The file makes it to the master's cache directory just fine, but is then garbled just like it was transferred as an ascii file instead of binary. The md5sums of the 2 files don't match anymore.
a4e1db13bef270c726cf30cfbabcbf19 /tmp/base.tar.gz
86dc81d6bbaf47c76366991f9b0466ae /var/cache/salt/master/minions/minion1/files/tmp/base.tar.gz

[basepi]

I think this issue has just been lost in the shuffle. Thanks for bumping it, it's still on our radar, just so many issues currently! =)

[terminalmage]

I've found and fixed the problem. That was a tricky one, but once I tracked it down the solution was pretty easy.