Python 2 support

[avosirenfal]

Has newtdb dropped Python 2 support? I get a serialization error with a very simple test case under Python 2, but not Python 3:

import logging
logging.basicConfig()

import newt.db
from BTrees._OOBTree import OOBTree

uplinkdb_con = newt.db.connection('postgresql://svelte@127.0.0.1/svelte')

class Profile(newt.db.Persistent):
	def __init__(self):
		self.rank = 'test'

root = uplinkdb_con.root
root.profiles = OOBTree()
root.profiles['somekey'] = Profile()
uplinkdb_con.commit()

Under Python 2 (Python 2.7.14, PostgreSQL 9.6.6) this causes:

ERROR:newt.db.jsonpickle:Failed pickle load, oid: 0, pickle starts: '\x80\x03cpersistent.mapping\nPersistentMapping\nq\x01.\x80\x03}q\x02U\x04'
Traceback (most recent call last):
  File "/home/svelte/venv/lib/python2.7/site-packages/newt/db/jsonpickle.py", line 509, in __call__
    state = unpickler.load()
  File "/home/svelte/venv/lib/python2.7/site-packages/newt/db/jsonpickle.py", line 227, in load
    for op, arg, pos in self.ops:
  File "/usr/lib64/python2.7/pickletools.py", line 1851, in genops
    code))
ValueError: at position 28, opcode 'C' unknown

Under Python 3 (Python 2 (Python 3.6.3, PostgreSQL 9.6.6) I get no output as expected. Everything looks correct in psql:

svelte=> select * from newt;
 zoid |              class_name              |                                       ghost_pickle                                       |                                            state
------+--------------------------------------+------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------
    0 | persistent.mapping.PersistentMapping | \x80036370657273697374656e742e6d617070696e670a50657273697374656e744d617070696e670a71002e | {"data": {"profiles": {"::": "persistent", "id": [1, "BTrees.OOBTree.OOBTree"], "::=>": 1}}}
    2 | __main__.Profile                     | \x8003635f5f6d61696e5f5f0a50726f66696c650a71002e                                         | {"rank": "test"}

The only virtualenv difference is Python 2 having trollius and futures.

Output of pip freeze for Python 2:

BTrees==4.5.0
cffi==1.11.5
futures==3.2.0
newt.db==0.9.0
perfmetrics==2.0
persistent==4.2.4.2
psycopg2==2.7.4
pycparser==2.18
RelStorage==2.1.0
six==1.11.0
transaction==2.2.1
trollius==2.2
zc.lockfile==1.3.0
ZConfig==3.2.0
zdaemon==4.2.0
ZEO==5.2.0
ZODB==5.4.0
zodbpickle==1.0
zope.interface==4.5.0

Output of pip freeze for Python 3:

BTrees==4.5.0
cffi==1.11.5
newt.db==0.9.0
perfmetrics==2.0
persistent==4.2.4.2
psycopg2==2.7.4
pycparser==2.18
RelStorage==2.1.0
six==1.11.0
transaction==2.2.1
zc.lockfile==1.3.0
ZConfig==3.2.0
zdaemon==4.2.0
ZEO==5.2.0
ZODB==5.4.0
zodbpickle==1.0
zope.interface==4.5.0

[jimfulton]

No, support for Python 2 hasn't been dropped.

ZODB 5.4 switched to pickle protocol 3 for Python 2 and it looks like this broke Newt.

The fix is straightforward and will happen soonish. :)

In the meantime, I suggest downgrading to ZODB 5.3 until we have a chance to fix this.

[avosirenfal]

That worked, thank you very much!