Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wal-e backup-push does't work with Python 3.6 #322

Closed
yteraoka opened this issue Mar 7, 2017 · 24 comments

Comments

Projects
None yet
@yteraoka
Copy link

commented Mar 7, 2017

-bash-4.2$ /opt/wal-e/bin/wal-e --aws-instance-profile --s3-prefix s3://****/wal-e backup-push $PGDATA
wal_e.main   INFO     MSG: starting WAL-E
        DETAIL: The subcommand is "backup-push".
        STRUCTURED: time=2017-03-07T15:00:32.823141-00 pid=14150
wal_e.operator.backup INFO     MSG: start upload postgres version metadata
        DETAIL: Uploading to s3://****/wal-e/basebackups_005/base_0000000100000002000000FE_00000040/extended_version.txt.
        STRUCTURED: time=2017-03-07T15:00:35.386069-00 pid=14150
wal_e.operator.backup INFO     MSG: postgres version metadata upload complete
        STRUCTURED: time=2017-03-07T15:00:35.458788-00 pid=14150
wal_e.worker.upload INFO     MSG: beginning volume compression
        DETAIL: Building volume 0.
        STRUCTURED: time=2017-03-07T15:00:35.585929-00 pid=14150
Traceback (most recent call last):
  File "/opt/wal-e/lib64/python3.6/site-packages/gevent/greenlet.py", line 536, in run
    result = self._run(*self.args, **self.kwargs)
  File "/opt/wal-e/lib64/python3.6/site-packages/wal_e/worker/upload.py", line 97, in __call__
    tpart.tarfile_write(pl.stdin)
  File "/opt/wal-e/lib64/python3.6/site-packages/wal_e/tar_partition.py", line 315, in tarfile_write
    self._padded_tar_add(tar, et_info)
  File "/opt/wal-e/lib64/python3.6/site-packages/wal_e/tar_partition.py", line 242, in _padded_tar_add
    tar.addfile(et_info.tarinfo, f)
  File "/usr/lib64/python3.6/tarfile.py", line 1973, in addfile
    copyfileobj(fileobj, self.fileobj, tarinfo.size, bufsize=bufsize)
TypeError: copyfileobj() got an unexpected keyword argument 'bufsize'
Wed Mar  8 00:00:35 2017 <Greenlet at 0x7f6121e8e5a0: <wal_e.worker.upload.PartitionUploader object at 0x7f6121e9f940>([ExtendedTarInfo(submitted_path='/var/lib/pgsql/9.)> failed with TypeError

wal_e.operator.backup WARNING  MSG: blocking on sending WAL segments
        DETAIL: The backup was not completed successfully, but we have to wait anyway.  See README: TODO about pg_cancel_backup
        STRUCTURED: time=2017-03-07T15:00:35.701163-00 pid=14150
NOTICE:  pg_stop_backup complete, all required WAL segments have been archived
wal_e.main   CRITICAL MSG: An unprocessed exception has avoided all error handling
        DETAIL: Traceback (most recent call last):
          File "/opt/wal-e/lib64/python3.6/site-packages/wal_e/cmd.py", line 627, in main
            pool_size=args.pool_size)
          File "/opt/wal-e/lib64/python3.6/site-packages/wal_e/operator/backup.py", line 197, in database_backup
            **kwargs)
          File "/opt/wal-e/lib64/python3.6/site-packages/wal_e/operator/backup.py", line 504, in _upload_pg_cluster_dir
            pool.join()
          File "/opt/wal-e/lib64/python3.6/site-packages/wal_e/worker/upload_pool.py", line 120, in join
            self._wait()
          File "/opt/wal-e/lib64/python3.6/site-packages/wal_e/worker/upload_pool.py", line 65, in _wait
            raise val
          File "/opt/wal-e/lib64/python3.6/site-packages/gevent/greenlet.py", line 536, in run
            result = self._run(*self.args, **self.kwargs)
          File "/opt/wal-e/lib64/python3.6/site-packages/wal_e/worker/upload.py", line 97, in __call__
            tpart.tarfile_write(pl.stdin)
          File "/opt/wal-e/lib64/python3.6/site-packages/wal_e/tar_partition.py", line 315, in tarfile_write
            self._padded_tar_add(tar, et_info)
          File "/opt/wal-e/lib64/python3.6/site-packages/wal_e/tar_partition.py", line 242, in _padded_tar_add
            tar.addfile(et_info.tarinfo, f)
          File "/usr/lib64/python3.6/tarfile.py", line 1973, in addfile
            copyfileobj(fileobj, self.fileobj, tarinfo.size, bufsize=bufsize)
        TypeError: copyfileobj() got an unexpected keyword argument 'bufsize'

        STRUCTURED: time=2017-03-07T15:00:37.771104-00 pid=14150
-bash-4.2$

It seems to be due to change of tarfile.py.
https://github.com/python/cpython/blob/3.5/Lib/tarfile.py
https://github.com/python/cpython/blob/3.6/Lib/tarfile.py

bufsize is always None?

$ uname -a
Linux ip-172-31-28-236.ap-northeast-1.compute.internal 3.10.0-514.10.2.el7.x86_64 #1 SMP Fri Mar 3 00:04:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Python 3.6 installed from IUC repository.

$ yum list installed | grep python36
python36u.x86_64                           3.6.0-2.ius.centos7         @ius
python36u-libs.x86_64                      3.6.0-2.ius.centos7         @ius
python36u-pip.noarch                       9.0.1-1.ius.centos7         @ius
python36u-setuptools.noarch                32.3.1-1.ius.centos7        @ius
@fdr

This comment has been minimized.

Copy link
Member

commented Mar 13, 2017

Thanks for the bug report.

Looks like Python 3.6 added a good feature, and that broke my workaround: instead of having to do my weird monkeypatch thing in copyfileobj.py, we might be able set the bufsize on a tarfile object directly (copybufsize).

How would you feel about patching this so that WAL-E's copyfileobj is not used on Python >= 3.6, and instead a large buffer set for tarfile's copyfileobj?

@fdr fdr added the bug label Mar 13, 2017

@yteraoka

This comment has been minimized.

Copy link
Author

commented Mar 19, 2017

Thanks.
That proposal sounds good to me.
I tried deleting the patch.
It worked.

bodymindarts added a commit to starkandwayne-attic/habitat-plans that referenced this issue May 9, 2017

@brikou

This comment has been minimized.

Copy link

commented Sep 5, 2017

Hello!
As written in readme, v1.1.0 is tested against python v3.6... sadly it still doesn't work with python 3.6, do you have any ETA for this fix?
Thanks a lot

@yteraoka

This comment has been minimized.

Copy link
Author

commented Sep 7, 2017

@brikou
Hi,
I tried that a half year ago. I did just simple test, and I have completely forgot that.
Before I tried that, the system began with python 3.5 now.
I will try again, so Please give me some time.

@yteraoka

This comment has been minimized.

Copy link
Author

commented Sep 8, 2017

@brikou
I tried again.

Following patch worked with Python 3.6 and 3.5. Could you please verify that this patch works?

--- /tmp/cmd.py.orig    2017-01-03 21:00:11.000000000 +0000
+++ /opt/wal-e/lib/python3.6/site-packages/wal_e/cmd.py 2017-09-08 10:31:53.567664129 +0000
@@ -589,7 +589,8 @@
         backup_cxt = configure_backup_cxt(args)

         if subcommand == 'backup-fetch':
-            monkeypatch_tarfile_copyfileobj()
+            if not (sys.version_info.major >= 3 and sys.version_info.minor >= 6):
+                monkeypatch_tarfile_copyfileobj()

             external_program_check([LZOP_BIN])
             backup_cxt.database_fetch(
@@ -601,7 +602,8 @@
         elif subcommand == 'backup-list':
             backup_cxt.backup_list(query=args.QUERY, detail=args.detail)
         elif subcommand == 'backup-push':
-            monkeypatch_tarfile_copyfileobj()
+            if not (sys.version_info.major >= 3 and sys.version_info.minor >= 6):
+                monkeypatch_tarfile_copyfileobj()

             if args.while_offline:
                 # we need to query pg_config first for the
@vmihailenco

This comment has been minimized.

Copy link

commented Oct 16, 2017

Ping. wal-e is still broken with Python 3.6.

@yteraoka

This comment has been minimized.

Copy link
Author

commented Oct 16, 2017

hmm.. 😕
My tested environment below.

$ cat /etc/redhat-release 
CentOS Linux release 7.4.1708 (Core)

$ yum list installed | grep postgresql
postgresql96.x86_64                         9.6.5-1PGDG.rhel7          @pgdg96  
postgresql96-libs.x86_64                    9.6.5-1PGDG.rhel7          @pgdg96  
postgresql96-server.x86_64                  9.6.5-1PGDG.rhel7          @pgdg96

$ yum list installed | grep python36
python36u.x86_64                            3.6.2-1.ius.centos7        @ius     
python36u-libs.x86_64                       3.6.2-1.ius.centos7        @ius     
python36u-pip.noarch                        9.0.1-1.ius.centos7        @ius     
python36u-setuptools.noarch                 33.1.1-1.ius.centos7       @ius

Installing WAL-E (S3 storage only)

# python3.6 -m venv /opt/wal-e
# /opt/wal-e/bin/pip install wal-e[aws]==1.1.0b1
# /opt/wal-e/bin/wal-e version
1.1.0b1

then apply the patch that disable monkeypatch_tarfile_copyfileobj() to cmd.py.

Which version do you use?
What error message displayed?

@vmihailenco

This comment has been minimized.

Copy link

commented Oct 16, 2017

@yteraoka I only mean that master branch is still broken. Your patch is probably working, but it would be nice if we get it into master branch...

reset added a commit to habitat-sh/core-plans that referenced this issue Oct 16, 2017

Lock wal-e to required version of Python
wal-e/wal-e#322

Signed-off-by: Jamie Winsor <jamie@vialstudios.com>
@h4k1m0u

This comment has been minimized.

Copy link

commented Nov 11, 2017

I got the same error as well when I updated to Python3.6 on FreeBSD11.
The version of Wal-e installed is 1.1.0.

@technion

This comment has been minimized.

Copy link

commented Nov 23, 2017

I can confirm that with 1.1.0 and your patch applied, the issue is resolved for me. It would be great to see this committed.

@loganb

This comment has been minimized.

Copy link

commented Jan 9, 2018

Hello all,

I'm getting this same failure using Python 3.6.4 and Wal-e 1.1.0b1 on OS X. I'm having trouble following the thread here. Is there a working workaround? Thanks!

@h4k1m0u

This comment has been minimized.

Copy link

commented Jan 9, 2018

@loganb There's a patch (proposed by @yteraoka) but I'm not sure how to apply it.
We're still waiting for that patch to be included in the master branch.

h4k1m0u added a commit to h4k1m0u/wal-e that referenced this issue Feb 18, 2018

Fix issue with copyfileobj() on Python36
copyfileobj() got an unexpected keyword argument 'bufsize'
See [wal-e#332](wal-e#322) and [wal-e#376](wal-e#376)

h4k1m0u added a commit to h4k1m0u/wal-e that referenced this issue Feb 18, 2018

Fix issue with copyfileobj() on Python36
copyfileobj() got an unexpected keyword argument 'bufsize'
See [wal-e#332](wal-e#322) and [wal-e#376](wal-e#376)

h4k1m0u added a commit to h4k1m0u/wal-e that referenced this issue Feb 19, 2018

Fix issue with copyfileobj() on Python36
copyfileobj() got an unexpected keyword argument 'bufsize'
See [wal-e#332](wal-e#322) and [wal-e#376](wal-e#376)
@markdavidburke

This comment has been minimized.

Copy link

commented May 16, 2018

Can confirm that this the above patch is working with the latest version of wal-e.

@fdr

This comment has been minimized.

Copy link
Member

commented May 16, 2018

@deverant any objections to merging this one? Patch seems fine to me on inspection.

@deverant

This comment has been minimized.

Copy link
Member

commented Jun 5, 2018

@fdr Looks good to me 👍

@deverant deverant closed this in #384 Jun 5, 2018

@thorsten-plause

This comment has been minimized.

Copy link

commented Jun 18, 2018

@fdr or @technion Any chance one of you can prepare a release with the python 3.6 support? Thank you!

@technion

This comment has been minimized.

Copy link

commented Jun 19, 2018

@thorsten-plause I am not a committer here.

@nimda7

This comment has been minimized.

Copy link

commented Jun 19, 2018

@thorsten-plause install it from master branch directly
pip install git+https://github.com/wal-e/wal-e.git

@thorsten-plause

This comment has been minimized.

Copy link

commented Jun 20, 2018

@nimda7 I am deploying it in a docker container, so using git is much harder. However, I just checked it out locally and copied the repo into my docker container for installation and that seems to work.
But still might be good to cut a release, the last release was done >6 months back.

@duanhongyi

This comment has been minimized.

Copy link

commented Nov 19, 2018

Can you publish an updated version? Even if it's just to solve this bug.

anonymouzz added a commit to innoteq/wal-e that referenced this issue Jun 6, 2019

@candysmurf

This comment has been minimized.

Copy link

commented Jun 27, 2019

@yteraoka, I ran into the same error. I need to make wale work inside a Docker container. Any help is greatly appreciated.

@yteraoka

This comment has been minimized.

Copy link
Author

commented Jun 27, 2019

@candysmurf I think this issue is resolved in #384 with a better way. But, It's not yet released on PyPI. You can install from github like this: pip install git+https://github.com/wal-e/wal-e.git@master. Be careful other changes in master branch. v1.1.0...master

@candysmurf

This comment has been minimized.

Copy link

commented Jun 27, 2019

@yteraoka, thanks for your quick help. I'll give it a try.

@candysmurf

This comment has been minimized.

Copy link

commented Jun 27, 2019

@yteraoka, yes, it worked for me. Thanks for your help! Do you know when this fix will be released?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.