s3 pillar fails on python3

[rklaren]

Description of Issue

Configuring an S3 pillar and doing a pillar.items yields the following stack strace:

2019-09-19 21:56:46,081 [salt.pillar      :984 ][ERROR   ][5957] Exception caught loading ext_pillar 's3':
  File "/usr/lib/python3/dist-packages/salt/pillar/__init__.py", line 974, in ext_pillar
    key)
  File "/usr/lib/python3/dist-packages/salt/pillar/__init__.py", line 893, in _external_pillar_data
    ext = self.ext_pillars[key](self.minion_id, pillar, **val)
  File "/usr/lib/python3/dist-packages/salt/pillar/s3.py", line 162, in ext_pillar
    metadata = _init(s3_creds, bucket, multiple_env, environment, prefix, s3_cache_expire)
  File "/usr/lib/python3/dist-packages/salt/pillar/s3.py", line 220, in _init
    environment, prefix)
  File "/usr/lib/python3/dist-packages/salt/pillar/s3.py", line 347, in _refresh_buckets_cache_file
    pickle.dump(metadata, fp_)

Setup

The most simple s3 pillar setup.

Steps to Reproduce Issue

Configure S3 pillar, refer to it, things go boom.

Versions Report

Salt Version:
           Salt: 2019.2.0
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: unknown
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: 2.0.3
      gitpython: 2.1.8
          ioflo: Not Installed
         Jinja2: 2.10
        libgit2: 0.26.0
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.26.2
         Python: 3.6.8 (default, Aug 20 2019, 17:12:48)
   python-gnupg: 0.4.1
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: 2.0.3
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5
 
System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-1047-aws
         system: Linux
        version: Ubuntu 18.04 bionic

The minion I used for testing is the master with the same versions.

The fix is to open the cache file in binary write mode:

    log.debug('Writing S3 buckets pillar cache file %s')

    with salt.utils.files.fopen(cache_file, 'wb') as fp_:
        pickle.dump(metadata, fp_)

I'll make a PR after some more testing.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[rklaren]

I reckon it's a bit early to close this bug.

[stale]

Thank you for updating this issue. It is no longer marked as stale.