TL;DR Use zodbupdate.
Data.fs which was created under Python 2 cannot be opened under Python 3. This is prevented by using a different magic code in the first bytes of the file. This is done on purpose because
str has a different meaning for the two Python versions: Under Python 2 a
str is a container for characters with an arbitrary encoding (aka
bytes). Python 3 knows
str as a text datatype which was called
unicode in Python 2. Trying to load a
str object in Python 3 which actually contains binary data will fail. It has to be
bytes is an alias for
str in Python 2 which means Python 2 replaces
str making is impossible to give Python 3 the class it expects for binary data. A Python 2
str with an arbitrary encoding will break, too.
Data.fs has to be migrated: each
str which actually contains
bytes has to be converted into a
zodbpickle.binary object which deserialises as
bytes under Python 3. The
str objects actually containing text have to be decoded to
unicode. There are currently two tools which claim that they are able to do such a migration:
- zodb.py3migrate was already written at Berlin Strategic sprint in 2016, but it was never able to prove that it can do what it claims: At the time when it was written there was no Zope which could run on Python 3. Now as we have Zope 4 running on Python 3 it does not seem to do its conversion job quite well: I was able to migrate a toy database but had to catch an unpickling error.
- zodbupdate was enriched by a Python 3 migration. A big thank you to Sylvain Viollon and the developers at Minddistrict! It has proven its claims! At the Zope 4 welcome sprint I was able to migrate a
Zope 2.13running on Python 2 to
Zope 4running on Python 3.
- Migrate your Zope application to
Zope 4. (
zodbupdaterequires at least
ZODB 4which is not the default ZODB version of
Zope 2.13) — For my toy database containing only a file object and an image this was no problem.
Zope 4is starting with such a database. It might show some broken objects because Zope no longer depends on some previous core packages like
Products.Sessions. If your application needs those packages you should add them to your Zope environment.
zodbupdatehas to be installed into the
Zope 4environment so it can access the Python classes. (It has to read the pickles in the ZODB.)
- There needs to be an
setup.pyfor each package which contains persistent Python classes. The entry point has to be named
"zodbupdate.decode"and needs to point to a dictionary mapping paths to
strattributes to a conversion (
bytesresp. a specific encoding). For Details see the migration documentation of zodbupdate. I prepared a branch of
Zope 4which contains this configuration dictionary for
OFS.File, see zopefoundation/Zope#285.
zodbupdate --pack --convert-py3on the
Data.fsusing Python 2.
- Copy the
Data.fsover to the
Zope 4instance running on Python 3.
Data.fs.indexwill be discarded at the first start. (There is an error message telling that it cannot be read.)
- Enjoy the contents of the
Data.fsrunning on Python 3.
It is possible (proven for a toy database) to migrate a
Zope 2.13 (Python 2) to
Zope 4 (Python 3).
zodbupdate is the way to go. Although it cannot do the migration completely autonomously the developers of Python packages can provide migration configuration in their packages which can be used in the migration step so the configuration has only to be written once.
zodb.py3migrate has an analysis step which shows the attribute names where the
str objects are stored. (This could be added to
zodbupdate, so do not expect that there will be two tools trying to achieve the same goal.)
mdtools.relstorage contains a
relstorage variant of
zodbupdate which claims to be much faster on
relstorage as it can leverage parallelism.
The pull request containing the migration strategy (zopefoundation/Zope#285) has to be extended for the other persistent classes in Zope. There have to be alike changes in all packages providing persistent classes.