Migrating from Zope 2.13 to 4.0b2

TL;DR: There are some rough edges when migrating an existing Data.fs and needed Python code from Zope 2.13 to Zope 4 but there is nothing what cannot be solved.

The story

The German company PerFact Innovation (www.perfact-innovation.de) has a customer product built on Zope 2.13. The code needed for the customisation to different customers is stored in the ZODB. They invited me to a workshop where we looked into the migration story to Zope 4. We used the just released Zope 4.0b2.

After half a day the core functions of the customer product where working including the switch from ZServer to WSGI. There were some rough edges we had to come around. I list them here as a reference for other people who start migrating their code to Zope 4.

We also poke around with a fresh Zope 4 installation on Python 3 to see what this will bring. It was nice to see that a DTMLMethod calling another one while mixing bytes and unicode  no longer leads to strange encoding problems but keeps bytes  as bytes.

Expect most of the bugs and some of the uglinesses listed below to be fixed in Zope 4.0b3.

Bugs

There are some bugs we could easily hack around in the workshop. Most of them are even fixed by now:

• Some objects cannot be loaded from the ZODB because their classes used to be old-style classes but in Zope 4.0b2 they became new-style classes. Fixed in [zopefoundation/Zope#205]
• Python scripts cannot be called because the compiled code is expected on a different attribute. This requires re-compiling them. Fixed in [zopefoundation/Products.PythonScripts#12]
• Some modules seem no longer allowed to be used in through the web PageTemplates, e. g. Products.PythonScripts.standard. Fixed in [zopefoundation/Zope#209]
• Accessing METAL macros using the getitem style (e. g. metal:use-macro=“python:here.master.macros[‘foo’]“) is not allowed. Its Python class seems to be missing a security declaration. Reported in [zopefoundation/Zope#210]
• When using Python 3 it is not possible to create a new Script (Python) in the ZMI because the default script code is not Python 3 compatible. Already fixed in [zopefoundation/Products.PythonScripts#10] awaiting release.

Uglinesses

Some things are still a bit raw in Zope 4 but for a beta version it is possible to live with them:

• The ZODB root object has no longer a default view. Entering http://localhost:8080/ in a browser leads to a SiteError. You have to add an index_html object in the ZODB yourself. Reported in [zopefoundation/Zope#212]
• Broken objects (because the needed Python code is not installed) are rendered like a Folder in the ZMI. Reported in [zopefoundation/Zope#213]
• If you enter code containing a SyntaxError in a Script (Python) and saving this code, you will see an error page and your code does not get saved. Previously a warning was rendered. Reported in [zopefoundation/Products.PythonScripts#11]
• Zope issues a warning when starting. Reported in [zopefoundation/Zope#211]
• After installing Products.PythonScripts there are even more warnings at startup. Reported in [zopefoundation/Products.PythonScripts#13]

Breaking changes

We were facing some of the breaking changes in Zope 4, which will require further development if the customer product should behave and be developed like before:

• Support for WebDAV, XML-RPC and FTP has been moved to the ZServer package. This means a WSGI based installation does not provide support for these protocols. Maybe WSGI middlewares could be written to support them again individually.
• The products directive is no longer supported in the zope.conf (only via ZServer). This means that the contents of the Products directory inside the Zope instance directory have to become Python packages which can be installed via pip resp. zc.buildout.

Required changes

We were facing some required changes on different levels to get the code running:

• Installation: Some Python packages are no longer dependencies of Zope or they have been extracted to separate packages and Zope does not depend on them. They have to be installed separately when they are needed. These packages are:
  • Products.PythonScripts
  • Products.ExternalMethod
  • Products.BTreeFolder2
  • Products.ZCatalog
  • Products.StandardCacheManagers
  • Products.SiteErrorLog
  • Products.MailHost
  • Products.TemporaryFolder (also contains Products.ZODBMountPoint)
  • Products.Sessions (also containsProducts.Transience)
  • Products.MIMETools
  • Record
• Products code: Zope 4 no longer contains the Globals module. In Zope 2.13 it only contained imports to keep the backwards compatibility with older Zope releases. Globals is gone now. To find out how to change your imports look at the Globals module on the 2.13 branch.
• PageTemplates: PageTemplates are now rendered using Chameleon. This prevents you from using -- in HTML comments as it is not allowed there. (See also the HTML comments document of the WebDesignGroup.)
• Error handling: standard_error_message does not exist any more in Zope 4. It only supports exception views which have to be written in file system code. See Catching and rendering exceptions as a tutorial how to create exception views and even a hack to restore using standard_error_message.

Conclusion

It is not impossible to migrate an existing Data.fs to Zope 4 (while staying on Python 2). It is even easier if the through the web approach was used to create the Data.fs as only very little code has to be modified that way.

Please test your applications on Zope 4 during the beta phase (until fall 2018) so we will get a final release which will be ready for production usage. If you have questions we are here to help.