[WIP] Convert ZODB via zodbupdate to Python 3

[icemac]

The str objects stored in the ZODB have either to be converted to bytes or unicode.

This PR contains the necessary conversion table for zodbupdate.

Status: The conversion table is not yet complete, but conversion worked for a ZODB containing OFS File and OFS Image objects.

Calling the migration (has to be done in a Zope instance running Zope 4 on Python 2):
bin/zodbupdate --pack --convert-py3 -f var/Data.fs

[rbu]

For reference, @icemac wrote a blog article about this at https://blog.gocept.com/2018/06/07/migrate-a-zope-zodb-data-fs-to-python-3/

[rbu]

For reference, @icemac wrote a blog article about this at https://blog.gocept.com/2018/06/07/migrate-a-zope-zodb-data-fs-to-python-3/

[rbu]

I've been using this script to generate experimental migrations. Given three virtualenvs (Zope2-Python2, Zope4-Python2, Zope4-Python3), I'm creating objects in the first and check if they are still usable in the last.

#!/bin/bash

set -e
set -x

basedir=~/zope/instances
instance=$1

[[ -z $instance ]] && { echo "provide instance name as first arg" ; exit ; }

if [[ ! -d $basedir/$instance-2py2 ]] ; then
    echo "Creating zope-2py2 $instance"

    ~/.virtualenvs/zope-2py2/bin/mkzopeinstance --user admin:admin --dir $basedir/$instance-2py2
fi

$basedir/$instance-2py2/bin/runzope

if [[ ! -d $basedir/$instance-4py2 ]] ; then
    echo "Creating zope-4py2 $instance"

    ~/.virtualenvs/zope-4py2/bin/mkwsgiinstance --user admin:admin --dir $basedir/$instance-4py2/
    cp $basedir/$instance-2py2/var/Data.fs $basedir/$instance-4py2/var/
fi

~/.virtualenvs/zope-4py2/bin/runwsgi $basedir/$instance-4py2/etc/zope.ini

if [[ ! -d $basedir/$instance-4py3 ]] ; then
    echo "Creating zope-4py3 $instance"

    ~/.virtualenvs/zope-4py3/bin/mkwsgiinstance --user admin:admin --dir $basedir/$instance-4py3/
    cp $basedir/$instance-4py2/var/Data.fs $basedir/$instance-4py3/var/
    
    ~/.virtualenvs/zope-4py2/bin/zodb-py3migrate-analyze $basedir/$instance-4py3/var/Data.fs
    ~/.virtualenvs/zope-4py2/bin/zodbupdate \
        --pack --convert-py3 --verbose --file \
        $basedir/$instance-4py3/var/Data.fs
fi

~/.virtualenvs/zope-4py3/bin/runwsgi $basedir/$instance-4py3/etc/zope.ini

[rbu]

I've been using this script to generate experimental migrations. Given three virtualenvs (Zope2-Python2, Zope4-Python2, Zope4-Python3), I'm creating objects in the first and check if they are still usable in the last.

#!/bin/bash

set -e
set -x

basedir=~/zope/instances
instance=$1

[[ -z $instance ]] && { echo "provide instance name as first arg" ; exit ; }

if [[ ! -d $basedir/$instance-2py2 ]] ; then
    echo "Creating zope-2py2 $instance"

    ~/.virtualenvs/zope-2py2/bin/mkzopeinstance --user admin:admin --dir $basedir/$instance-2py2
fi

$basedir/$instance-2py2/bin/runzope

if [[ ! -d $basedir/$instance-4py2 ]] ; then
    echo "Creating zope-4py2 $instance"

    ~/.virtualenvs/zope-4py2/bin/mkwsgiinstance --user admin:admin --dir $basedir/$instance-4py2/
    cp $basedir/$instance-2py2/var/Data.fs $basedir/$instance-4py2/var/
fi

~/.virtualenvs/zope-4py2/bin/runwsgi $basedir/$instance-4py2/etc/zope.ini

if [[ ! -d $basedir/$instance-4py3 ]] ; then
    echo "Creating zope-4py3 $instance"

    ~/.virtualenvs/zope-4py3/bin/mkwsgiinstance --user admin:admin --dir $basedir/$instance-4py3/
    cp $basedir/$instance-4py2/var/Data.fs $basedir/$instance-4py3/var/
    
    ~/.virtualenvs/zope-4py2/bin/zodb-py3migrate-analyze $basedir/$instance-4py3/var/Data.fs
    ~/.virtualenvs/zope-4py2/bin/zodbupdate \
        --pack --convert-py3 --verbose --file \
        $basedir/$instance-4py3/var/Data.fs
fi

~/.virtualenvs/zope-4py3/bin/runwsgi $basedir/$instance-4py3/etc/zope.ini

[rbu]

With d7fcae1, you'll probably seeNoneType has no attribute decode style errors unless you have zopefoundation/zodbupdate#6 merged into zodbupdate,

[rbu]

With d7fcae1, you'll probably seeNoneType has no attribute decode style errors unless you have zopefoundation/zodbupdate#6 merged into zodbupdate,

[frisi]

just to let you know: i've updated the description of plone ticket dealing with ZODB migration for python3 (plone/Products.CMFPlone#2525) and most of the stuff there is important/relevant for pure zope applications too.

[frisi]

just to let you know: i've updated the description of plone ticket dealing with ZODB migration for python3 (plone/Products.CMFPlone#2525) and most of the stuff there is important/relevant for pure zope applications too.

[rbu]

I started writing some documentation. @icemac, I copied parts of your blog post. Hope that's ok.

[rbu]

I started writing some documentation. @icemac, I copied parts of your blog post. Hope that's ok.

[icemac]

@rbu That's okay. I am merging the PR to get it into the next Zope release.

[icemac]

@rbu That's okay. I am merging the PR to get it into the next Zope release.