Earl Zope was hoping to get the final permission for the Python 3 wonderland on the Saltlabs Sprint in Halle last week. He has made good progress in settling down:
- Zope claims compatibility with the newest Python standards (Python 3.7).
- Earl Zope now has new cloths for his administrative interface (called ZMI) as shown on the image above.
- Zope is playing nicer together with Prince Plone who is also migrating to the Python 3 wonderland
- Documentation got improved and at least partly updated to Zope 4.
- Zope got some other bug fixes.
But this was not enough for the immigration authorities to give Earl Zope the final permission. The following was missing:
- a working and proven story how to migrate Earl Zope’s treasures stored the
There was good progress towards this goal but it could not be fully reached. So gocept will have the honour to host another sprint maybe in May next year.
The new beta licence has the id
4.0b6 and can be achieved and used as described in the Zope documentation.
TL;DR Use zodbupdate.
Data.fs which was created under Python 2 cannot be opened under Python 3. This is prevented by using a different magic code in the first bytes of the file. This is done on purpose because
str has a different meaning for the two Python versions: Under Python 2 a
str is a container for characters with an arbitrary encoding (aka
bytes). Python 3 knows
str as a text datatype which was called
unicode in Python 2. Trying to load a
str object in Python 3 which actually contains binary data will fail. It has to be
bytes is an alias for
str in Python 2 which means Python 2 replaces
str making is impossible to give Python 3 the class it expects for binary data. A Python 2
str with an arbitrary encoding will break, too.
Data.fs has to be migrated: each
str which actually contains
bytes has to be converted into a
zodbpickle.binary object which deserialises as
bytes under Python 3. The
str objects actually containing text have to be decoded to
unicode. There are currently two tools which claim that they are able to do such a migration:
- zodb.py3migrate was already written at Berlin Strategic sprint in 2016, but it was never able to prove that it can do what it claims: At the time when it was written there was no Zope which could run on Python 3. Now as we have Zope 4 running on Python 3 it does not seem to do its conversion job quite well: I was able to migrate a toy database but had to catch an unpickling error.
- zodbupdate was enriched by a Python 3 migration. A big thank you to Sylvain Viollon and the developers at Minddistrict! It has proven its claims! At the Zope 4 welcome sprint I was able to migrate a
Data.fs created on
Zope 2.13 running on Python 2 to
Zope 4 running on Python 3.
- Migrate your Zope application to
Zope 4. (
zodbupdate requires at least
ZODB 4 which is not the default ZODB version of
Zope 2.13) — For my toy database containing only a file object and an image this was no problem.
Zope 4 is starting with such a database. It might show some broken objects because Zope no longer depends on some previous core packages like
Products.Sessions. If your application needs those packages you should add them to your Zope environment.
zodbupdate has to be installed into the
Zope 4 environment so it can access the Python classes. (It has to read the pickles in the ZODB.)
- There needs to be an
setup.py for each package which contains persistent Python classes. The entry point has to be named
"zodbupdate.decode" and needs to point to a dictionary mapping paths to
str attributes to a conversion (
bytes resp. a specific encoding). For Details see the migration documentation of zodbupdate. I prepared a branch of
Zope 4 which contains this configuration dictionary for
OFS.File, see zopefoundation/Zope#285.
zodbupdate --pack --convert-py3 on the
Data.fs using Python 2.
- Copy the
Data.fs over to the
Zope 4 instance running on Python 3.
Data.fs.index will be discarded at the first start. (There is an error message telling that it cannot be read.)
- Enjoy the contents of the
Data.fs running on Python 3.
It is possible (proven for a toy database) to migrate a
Zope 2.13 (Python 2) to
Zope 4 (Python 3).
zodbupdate is the way to go. Although it cannot do the migration completely autonomously the developers of Python packages can provide migration configuration in their packages which can be used in the migration step so the configuration has only to be written once.
zodb.py3migrate has an analysis step which shows the attribute names where the
str objects are stored. (This could be added to
zodbupdate, so do not expect that there will be two tools trying to achieve the same goal.)
mdtools.relstorage contains a
relstorage variant of
zodbupdate which claims to be much faster on
relstorage as it can leverage parallelism.
The pull request containing the migration strategy (zopefoundation/Zope#285) has to be extended for the other persistent classes in Zope. There have to be alike changes in all packages providing persistent classes.