New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Convert ZODB via zodbupdate to Python 3 #285

Merged
merged 4 commits into from Oct 10, 2018

Conversation

3 participants
@icemac
Member

icemac commented May 18, 2018

The str objects stored in the ZODB have either to be converted to bytes or unicode.

This PR contains the necessary conversion table for zodbupdate.

Status: The conversion table is not yet complete, but conversion worked for a ZODB containing OFS File and OFS Image objects.

Calling the migration (has to be done in a Zope instance running Zope 4 on Python 2):
bin/zodbupdate --pack --convert-py3 -f var/Data.fs

@icemac icemac added this to To do in Zope 4 final release Jul 12, 2018

@icemac icemac moved this from To do to In progress in Zope 4 final release Jul 12, 2018

@icemac icemac added this to the 4.0 final milestone Sep 18, 2018

@icemac icemac added the enhancement label Sep 18, 2018

@rbu rbu self-assigned this Oct 1, 2018

@rbu

This comment has been minimized.

Show comment
Hide comment
@rbu
Member

rbu commented Oct 1, 2018

@rbu rbu closed this Oct 1, 2018

Zope 4 final release automation moved this from In progress to Done Oct 1, 2018

@rbu rbu reopened this Oct 1, 2018

Zope 4 final release automation moved this from Done to In progress Oct 1, 2018

@rbu

This comment has been minimized.

Show comment
Hide comment
@rbu

rbu Oct 2, 2018

Member

I've been using this script to generate experimental migrations. Given three virtualenvs (Zope2-Python2, Zope4-Python2, Zope4-Python3), I'm creating objects in the first and check if they are still usable in the last.

#!/bin/bash

set -e
set -x

basedir=~/zope/instances
instance=$1

[[ -z $instance ]] && { echo "provide instance name as first arg" ; exit ; }

if [[ ! -d $basedir/$instance-2py2 ]] ; then
    echo "Creating zope-2py2 $instance"

    ~/.virtualenvs/zope-2py2/bin/mkzopeinstance --user admin:admin --dir $basedir/$instance-2py2
fi

$basedir/$instance-2py2/bin/runzope

if [[ ! -d $basedir/$instance-4py2 ]] ; then
    echo "Creating zope-4py2 $instance"

    ~/.virtualenvs/zope-4py2/bin/mkwsgiinstance --user admin:admin --dir $basedir/$instance-4py2/
    cp $basedir/$instance-2py2/var/Data.fs $basedir/$instance-4py2/var/
fi

~/.virtualenvs/zope-4py2/bin/runwsgi $basedir/$instance-4py2/etc/zope.ini

if [[ ! -d $basedir/$instance-4py3 ]] ; then
    echo "Creating zope-4py3 $instance"

    ~/.virtualenvs/zope-4py3/bin/mkwsgiinstance --user admin:admin --dir $basedir/$instance-4py3/
    cp $basedir/$instance-4py2/var/Data.fs $basedir/$instance-4py3/var/
    
    ~/.virtualenvs/zope-4py2/bin/zodb-py3migrate-analyze $basedir/$instance-4py3/var/Data.fs
    ~/.virtualenvs/zope-4py2/bin/zodbupdate \
        --pack --convert-py3 --verbose --file \
        $basedir/$instance-4py3/var/Data.fs
fi

~/.virtualenvs/zope-4py3/bin/runwsgi $basedir/$instance-4py3/etc/zope.ini
Member

rbu commented Oct 2, 2018

I've been using this script to generate experimental migrations. Given three virtualenvs (Zope2-Python2, Zope4-Python2, Zope4-Python3), I'm creating objects in the first and check if they are still usable in the last.

#!/bin/bash

set -e
set -x

basedir=~/zope/instances
instance=$1

[[ -z $instance ]] && { echo "provide instance name as first arg" ; exit ; }

if [[ ! -d $basedir/$instance-2py2 ]] ; then
    echo "Creating zope-2py2 $instance"

    ~/.virtualenvs/zope-2py2/bin/mkzopeinstance --user admin:admin --dir $basedir/$instance-2py2
fi

$basedir/$instance-2py2/bin/runzope

if [[ ! -d $basedir/$instance-4py2 ]] ; then
    echo "Creating zope-4py2 $instance"

    ~/.virtualenvs/zope-4py2/bin/mkwsgiinstance --user admin:admin --dir $basedir/$instance-4py2/
    cp $basedir/$instance-2py2/var/Data.fs $basedir/$instance-4py2/var/
fi

~/.virtualenvs/zope-4py2/bin/runwsgi $basedir/$instance-4py2/etc/zope.ini

if [[ ! -d $basedir/$instance-4py3 ]] ; then
    echo "Creating zope-4py3 $instance"

    ~/.virtualenvs/zope-4py3/bin/mkwsgiinstance --user admin:admin --dir $basedir/$instance-4py3/
    cp $basedir/$instance-4py2/var/Data.fs $basedir/$instance-4py3/var/
    
    ~/.virtualenvs/zope-4py2/bin/zodb-py3migrate-analyze $basedir/$instance-4py3/var/Data.fs
    ~/.virtualenvs/zope-4py2/bin/zodbupdate \
        --pack --convert-py3 --verbose --file \
        $basedir/$instance-4py3/var/Data.fs
fi

~/.virtualenvs/zope-4py3/bin/runwsgi $basedir/$instance-4py3/etc/zope.ini
@rbu

This comment has been minimized.

Show comment
Hide comment
@rbu

rbu Oct 2, 2018

Member

With d7fcae1, you'll probably seeNoneType has no attribute decode style errors unless you have zopefoundation/zodbupdate#6 merged into zodbupdate,

Member

rbu commented Oct 2, 2018

With d7fcae1, you'll probably seeNoneType has no attribute decode style errors unless you have zopefoundation/zodbupdate#6 merged into zodbupdate,

@rbu rbu removed their assignment Oct 2, 2018

@frisi

This comment has been minimized.

Show comment
Hide comment
@frisi

frisi Oct 5, 2018

Member

just to let you know: i've updated the description of plone ticket dealing with ZODB migration for python3 (plone/Products.CMFPlone#2525) and most of the stuff there is important/relevant for pure zope applications too.

Member

frisi commented Oct 5, 2018

just to let you know: i've updated the description of plone ticket dealing with ZODB migration for python3 (plone/Products.CMFPlone#2525) and most of the stuff there is important/relevant for pure zope applications too.

icemac and others added some commits Oct 5, 2018

@rbu

This comment has been minimized.

Show comment
Hide comment
@rbu

rbu Oct 9, 2018

Member

I started writing some documentation. @icemac, I copied parts of your blog post. Hope that's ok.

Member

rbu commented Oct 9, 2018

I started writing some documentation. @icemac, I copied parts of your blog post. Hope that's ok.

@icemac icemac merged commit a910233 into master Oct 10, 2018

3 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
coverage/coveralls Coverage increased (+0.6%) to 79.943%
Details

Zope 4 final release automation moved this from In progress to Done Oct 10, 2018

@icemac icemac deleted the zodbupdate branch Oct 10, 2018

@icemac icemac referenced this pull request Oct 10, 2018

Open

Complete Python 2 -> 3 migration story #369

0 of 2 tasks complete
@icemac

This comment has been minimized.

Show comment
Hide comment
@icemac

icemac Oct 10, 2018

Member

@rbu That's okay. I am merging the PR to get it into the next Zope release.

Member

icemac commented Oct 10, 2018

@rbu That's okay. I am merging the PR to get it into the next Zope release.

@icemac icemac removed this from the 4.0 final milestone Oct 11, 2018

@icemac icemac added this to the 4.0b6 milestone Oct 11, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment