{"id":3350,"date":"2019-11-13T09:08:02","date_gmt":"2019-11-13T08:08:02","guid":{"rendered":"http:\/\/blog.gocept.com\/?p=3350"},"modified":"2019-11-13T09:10:47","modified_gmt":"2019-11-13T08:10:47","slug":"union-cms-released-on-python-3","status":"publish","type":"post","link":"https:\/\/blog.gocept.com\/2019\/11\/13\/union-cms-released-on-python-3\/","title":{"rendered":"union.cms released on Python 3"},"content":{"rendered":"\n

union.cms is a content management system which was once developed on Zope 2. It was one of the early adopters of the Five technology aka using Zope 3 components in Zope 2. Now it is one of the proud early adopters of Zope 4 on Python 3. It is used as CMS for large organisations.<\/p>\n\n\n\n

In this post we want to present our process how we tackled the migration and briefly discuss occurred issues. Our migration plan looked like the following:<\/p>\n\n\n\n

  1. Port union.cms to Zope 4 while still running on Python 2.7.<\/li>
  2. Rollout this version to production \u2013 this already has happened in December 2018, see union.cms launched on Zope\u00a04<\/a>.<\/li>
  3. Port the code and the tests to Python 3 while keeping it compatible with Python 2.7 \u2013 this was done without branching a dedicated Python 3 branch, instead with a continuous integration system running against Python 2.7 and 3.7 to ensure the possibility of a prompt deployment.<\/li>
  4. Have some releases to production during the migration phase \u2013 this included also releases with new features developed in parallel.<\/li>
  5. Migrate the ZODB based database to be readable by an application server running on Python 3 \u2013 Thank you to everyone who contributed to zodbupdate<\/a> which allowed to have a smooth migration, details see below.<\/li>
  6. Run manual tests on a staging system \u2013 by ourselves and by the customer to find edge cases not detected by the automatic tests. (There where only a few which where easy to reproduce and to fix.)<\/li>
  7. Rollout to production on Python 3 \u2013 this was done at the beginning of November 2019.<\/li>
  8. Drop the Python 2 support code \u2013 this is still open but of low risk.<\/li><\/ol>\n\n\n\n

    During the preparation of the project the migration of the Data.fs seemed to be the hardest part. There where no tools in the beginning and the migration had to be done because of the str<\/code> vs. binary<\/code> issues between Python 2 and 3. (For details on this topic see Migrate a Zope ZODB Data.fs to Python\u00a03<\/a>.)<\/p>\n\n\n\n

    Actually the hardest part was the migration from Zope 2 to Zope 4. There are some internal changes where it is not easy to see what to do to make our own code compatible. Additionally we updated all dependencies to the newest versions which support both Python 2 and 3 to prepare the final switch to Python 3.<\/p>\n\n\n\n

    The migration of the code to Python 3 was a lot of work. It included to change some dependencies to other packages which have already been ported to Python 3 instead of depending on unmaintained ones. But most of the dependencies were in a usable state. pylint<\/a> was used on Python 2 to detect code which will cause problems on Python 3. (This requires to use a pylint version older than version 2. We called it using pylint --py3k --disable=no-absolute-import src\/** setup.py<\/code>.) Most parts of the migration could be done automatically using modernize<\/a> leaving the more trickier ones for the developers.<\/p>\n\n\n\n

    The migration of the database ran smoothly. The only issue was hidden inside ZCatalog where some index contents were stored as binary<\/code> but str<\/code> was expected. This could be solved by creating and running a migration script. (Details see Products.ZCatalog#83<\/a>.)<\/p>\n\n\n\n

    The rollout to production went without problems even though databases of more than 10 GByte size had to be migrated. Thankfully it was possible to do the migration offline instead of being forced to do a live migration.<\/p>\n\n\n\n

    The whole migration project went about two years. We decided for a slower migration with at least some deployments to production to prove the already done steps in a live environment and to allow new features and bug fixes during the migration project. This approach went well, so we can suggest it for other migration projects.<\/p>\n\n\n\n

    By now union.cms runs live to Zope 4.1 using Python 3.7. It’s time to celebrate that Zope 4 on Python 3 can be used for actual projects in a live environment. \ud83c\udf89<\/p>\n","protected":false},"excerpt":{"rendered":"

    union.cms is a content management system which was once developed on Zope 2. It was one of the early adopters of the Five technology aka using Zope 3 components in Zope 2. Now it is one of the proud early adopters of Zope 4 on Python 3. It is used as CMS for large organisations. … Continue reading “union.cms released on Python 3”<\/span><\/a><\/p>\n","protected":false},"author":15344399,"featured_media":3354,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_newsletter_tier_id":0,"footnotes":"","jetpack_publicize_message":"union.cms released on Python 3 #Zope #Python3","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[10221],"tags":[8915,12071,5336674,663275160,72208,594099838],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2019\/11\/green-tree-python-1312700.jpg?fit=1200%2C863&ssl=1","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/pFP3y-S2","jetpack-related-posts":[{"id":3276,"url":"https:\/\/blog.gocept.com\/2018\/12\/19\/union-cms-launched-on-zope-4\/","url_meta":{"origin":3350,"position":0},"title":"union.cms launched on Zope 4","author":"Michael Howitz","date":"December 19, 2018","format":false,"excerpt":"union.cms is a content management system which was once developed on Zope 2. It was one of the early adopters of the Five technology: Using Zope 3 components in Zope 2. Now it is one of the proud early adopters of Zope 4. It is used as CMS for large\u2026","rel":"","context":"In "en"","block_context":{"text":"en","link":"https:\/\/blog.gocept.com\/category\/en\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2018\/12\/emotions-3419025_1920.jpg?fit=1200%2C600&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2018\/12\/emotions-3419025_1920.jpg?fit=1200%2C600&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2018\/12\/emotions-3419025_1920.jpg?fit=1200%2C600&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2018\/12\/emotions-3419025_1920.jpg?fit=1200%2C600&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2018\/12\/emotions-3419025_1920.jpg?fit=1200%2C600&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":3306,"url":"https:\/\/blog.gocept.com\/2019\/05\/10\/zope-roadmap\/","url_meta":{"origin":3350,"position":1},"title":"Zope roadmap","author":"Michael Howitz","date":"May 10, 2019","format":false,"excerpt":"Zope 4 is the successor of Zope 2.13 supporting both Python 2.7 and Python 3. After the release of its final version 4.0: Zope 2.13 will drop into \"security fixes only\" mode. It will stay in this mode as long as there is a supported Plone version using Zope 2.13:\u2026","rel":"","context":"In "en"","block_context":{"text":"en","link":"https:\/\/blog.gocept.com\/category\/en\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2019\/05\/abend-amerika-ausflug-844167.jpg?fit=1200%2C772&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2019\/05\/abend-amerika-ausflug-844167.jpg?fit=1200%2C772&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2019\/05\/abend-amerika-ausflug-844167.jpg?fit=1200%2C772&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2019\/05\/abend-amerika-ausflug-844167.jpg?fit=1200%2C772&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2019\/05\/abend-amerika-ausflug-844167.jpg?fit=1200%2C772&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":3225,"url":"https:\/\/blog.gocept.com\/2018\/05\/24\/zope-is-welcome-in-the-python-3-wonderland\/","url_meta":{"origin":3350,"position":2},"title":"Zope is welcome in the Python 3 wonderland!","author":"Michael Howitz","date":"May 24, 2018","format":false,"excerpt":"Earl Zope already got the beta permission to stay in the Python 3 wonderland some months ago. His current objective is to help old friends to come to the Python 3 wonderland and to make new friends. He has to build trust in his will and ability to stay in\u2026","rel":"","context":"In "en"","block_context":{"text":"en","link":"https:\/\/blog.gocept.com\/category\/en\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2018\/05\/zope-4-welcome-sprint.jpg?fit=1200%2C637&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2018\/05\/zope-4-welcome-sprint.jpg?fit=1200%2C637&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2018\/05\/zope-4-welcome-sprint.jpg?fit=1200%2C637&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2018\/05\/zope-4-welcome-sprint.jpg?fit=1200%2C637&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2018\/05\/zope-4-welcome-sprint.jpg?fit=1200%2C637&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":3312,"url":"https:\/\/blog.gocept.com\/2019\/05\/10\/celebration-zope-4-final-release\/","url_meta":{"origin":3350,"position":3},"title":"Celebration: Zope 4 final release","author":"Michael Howitz","date":"May 10, 2019","format":false,"excerpt":"TL;DR: Zope 4 beta phase ended, final version released! After hard, long years of preparation Earl Zope now finally made it to get a permanent license for the Python 3 wonderland: In September 2016 almost 20 people started with the reanimation of Zope at the Zope Resurrection sprint. This marked\u2026","rel":"","context":"In "en"","block_context":{"text":"en","link":"https:\/\/blog.gocept.com\/category\/en\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2019\/05\/fireworks-945386_1920.jpg?fit=1200%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2019\/05\/fireworks-945386_1920.jpg?fit=1200%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2019\/05\/fireworks-945386_1920.jpg?fit=1200%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2019\/05\/fireworks-945386_1920.jpg?fit=1200%2C800&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2019\/05\/fireworks-945386_1920.jpg?fit=1200%2C800&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":2102,"url":"https:\/\/blog.gocept.com\/2017\/02\/21\/zope-at-the-turnpike-of-the-python-3-wonderland\/","url_meta":{"origin":3350,"position":4},"title":"Zope at the turnpike of the Python 3 wonderland","author":"Michael Howitz","date":"February 21, 2017","format":false,"excerpt":"A little tale Once upon the time there was an earl named Zope II. He\u00a0lived happily in a land called Python 2. Since some years there where rumours that a\u00a0huge\u00a0disaster would\u00a0hit the country. The people ironically used to call it \"sunset\". Prophets arose and said that 2020 would be the\u2026","rel":"","context":"In "en"","block_context":{"text":"en","link":"https:\/\/blog.gocept.com\/category\/en\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2017\/02\/wallpapers-bridge.jpg?fit=1200%2C800&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2017\/02\/wallpapers-bridge.jpg?fit=1200%2C800&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2017\/02\/wallpapers-bridge.jpg?fit=1200%2C800&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2017\/02\/wallpapers-bridge.jpg?fit=1200%2C800&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2017\/02\/wallpapers-bridge.jpg?fit=1200%2C800&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":3229,"url":"https:\/\/blog.gocept.com\/2018\/06\/07\/migrate-a-zope-zodb-data-fs-to-python-3\/","url_meta":{"origin":3350,"position":5},"title":"Migrate a Zope ZODB Data.fs to Python 3","author":"Michael Howitz","date":"June 7, 2018","format":false,"excerpt":"TL;DR Use\u00a0zodbupdate. Problem A ZODB\u00a0Data.fs\u00a0which was created under Python 2 cannot be opened under Python 3. This is prevented by using a different magic code in the first bytes of the file. This is done on purpose because str\u00a0has a different meaning for the two Python versions: Under Python 2\u2026","rel":"","context":"In "en"","block_context":{"text":"en","link":"https:\/\/blog.gocept.com\/category\/en\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2018\/06\/spring-3383890_1280.jpg?fit=1200%2C797&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2018\/06\/spring-3383890_1280.jpg?fit=1200%2C797&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2018\/06\/spring-3383890_1280.jpg?fit=1200%2C797&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2018\/06\/spring-3383890_1280.jpg?fit=1200%2C797&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/blog.gocept.com\/wp-content\/uploads\/2018\/06\/spring-3383890_1280.jpg?fit=1200%2C797&ssl=1&resize=1050%2C600 3x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/posts\/3350"}],"collection":[{"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/users\/15344399"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/comments?post=3350"}],"version-history":[{"count":11,"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/posts\/3350\/revisions"}],"predecessor-version":[{"id":3365,"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/posts\/3350\/revisions\/3365"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/media\/3354"}],"wp:attachment":[{"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/media?parent=3350"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/categories?post=3350"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/tags?post=3350"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}