english articles
The gocept team will join PyConWeb 2017 in Munich from 27th to 28th of May – hey, this is is less than one week from now! It seems that there are still tickets available.
I myself will present RestrictedPython – or how to port to Python 3 without porting dependencies on Saturday at 3 p. m.
See you in Munich!
The sprint days were really busy for Earl Zope II and the people helping him with the Python 3 wonderland immigration authorities.
Besides working on Zope there was other ongoing work:
His majesty Earl Zope II says a warm “Thank you!” to all who helped him to start his new live in Python 3 wonderland. There is still enough work to be done so he can live there and having all the comfort and stability of Python 3. See you on the next sprint!
Welcome to the Zope 2 Resurrection Sprint in Halle (Saale), Germany. We hope you enjoyed the time since the Last call for take off to the Python 3 wonderland.
We already achieved some things:
We discussed the following topics:
bin/test instead of python setup.py test to run the tests as the latter one is no longer liked by the Python Packaging Authority folks.Last information for Zope 2 Resurrection Sprint
We are approaching the Zope 2 Resurrection Sprint and hope that all those who are willing to help earl Zope II on his endeavor to port his realm have already prepared there horses and packed the necessary equipment to arrive in Halle (Saale), Germany.
To help with the preparations we have set up some means of communication:
In the Etherpad we have collected a fundamental set of obstacles that the immigration authority of Python 3 wonderland send to earl Zope II via a mounted messenger. If there are additional problems we need to solve with the immigration or other authorities, feel free to point those out in the pad.
During the sprint we will have an owl waiting for messages in the #sprint channel on irc.freenode.net, so additional information and questions can be placed there.
In general the gates of the gocept manor in Halle (Saale) are open from 8:00 till 18:00 during the sprint for the squires to help earl Zope II. There will be some refreshments in the morning (8:00 – 9:00) and during lunch time (12:00 – 13:00) in order to keep everyone happy and content.
Apart from that, there will be some fixed points in time to meet:
We are looking forward to the sprint and hope to overcome the remaining migration problems of earl Zope II.
Earlier this year there was a sprint in Innsbruck, Austria. We made progress in porting Zope to Python 3 by working on RestrictedPython. After this sprint RestrictedPython no longer seems to be a blocker to port the parts of Zope which rely on RestrictedPython to Python 3.
See the full sprint report on the plone.org website.
We will work further on pushing Zope towards the Python 3 wonderland on the Zope 2 Resurrection Sprint in Halle/Saale, Germany at gocept in the first week of May 2017. You are welcome to join us on site or remote.
Photo copyright: Christine Baumgartner
A little tale
Once upon the time there was an earl named Zope II. He lived happily in a land called Python 2. Since some years there where rumours that a huge disaster would hit the country. The people ironically used to call it “sunset”. Prophets arose and said that 2020 would be the year where this disaster would finally happen.
Zope II got anxious about his future and the future of his descendants. But there were some brave people who liked Zope II and told him of the Python 3 wonderland. A land of eternal joy and happiness without “sunset” disasters and with no problems at all. It seemed like a dream to Zope II – too nice to be true?
After some research it became clear that the Python 3 wonderland was real – not completely as advertised by the people but nice to live in. So Zope II set the goal to settle down in the Python 3 wonderland before the “sunset” disaster would happen.
But this was not as easy as it seemed to be. The immigrant authority told Zope II that he is “not compatible” with the Python 3 wonderland and that he needs “to be ported” to be able to breath the special Python 3 air.
As if this was not enough: As an earl Zope II was not able to migrate without his staff, the many people he relied on for his daily procedures. Some of them have been found to be “ported” already thus they were already “compatible” for the new country. But there was one old, but very important servant of Zope II named RestrictedPython. Zope II could not live without him – but he got told that he never will be “compatible”. The authority even required a “complete rewrite”.
The Python 3 wonderland seemed so near but is was so difficult to reach. But there where so many people who helped Zope II and encouraged him not to give up. Eventually it seemed to be possible for Zope II to get beyond the turnpike into the wonderful land called Python 3.
Back in reality
We are the people who like Zope II and help him to get past the turnpike into Python 3. Since the Alpine City Sprint earlier this month RestrictedPython is no longer a blocker for other packages depending on it to be ported.
Come, join us porting the remaining dependencies of Zope and Zope itself to Python 3. There will be a Zope 2 Resurrection Sprint in May this year in Halle (Saale), Germany at gocept. Join us on site or remote.
The biggest blocker to port Zope to Python 3 is RestrictedPython.
What is RestrictedPython?
It is a library used by Zope to restrict Python code at instruction level to a bare minimum of trusted functionality. It parses and filters the code for not allowed constructs (such as open()) and adds wrappers around each access on attributes or items. These wrappers can be used by Zope to enforce access control on objects in the ZODB without requiring manual checks in the code.
Why is RestrictedPython needed?
Zope allows writing Python code in the Zope management interface (ZMI) using a web browser (“through the web” aka TTW). This code is stored in the ZODB. The code is executed on the server. It would be dangerous to allow a user to execute arbitrary code with the rights of the web server process. That’s why the code is filtered through RestrictedPython to make sure this approach is not a complete security hole.
RestrictedPython is used in many places of Zope as part of its security model. An experiment on the Zope Resurrection Sprint showed that it would be really hard to create a Zope version which does not need RestrictedPython thus removing the TTW approach.
What is the problem porting RestrictedPython to Python 3?
RestrictedPython relies on the compiler package of the Python standard library. This package no longer exists in Python 3 because it was poorly documented, unmaintained and out of sync with the compiler Python uses itself. (There are whisperings that it was only kept because of Zope.)
Since Python 2.6 there is a new ast module in the Python standard library which is not a direct replacement for compiler. There is no documentation how to replace compiler by ast.
What is the current status?
Several people already worked on various Plone and Zope sprints and mostly in their spare time on a Python 3 branch of RestrictedPython to find out how this package works and to start porting some of its functionality as a proof of concept. It seems to be possible to use ast as the new base for RestrictedPython. Probably the external API of RestrictedPython could be kept stable. But packages using or extending some of the internals of RestrictedPython might need to be updated as well.
What are the next steps?
Many Zope and Plone packages depend on RestrictedPython directly (like AccessControl or Products.ZCatalog) or indirectly (like Products.PythonScripts, plone.app.event or even plone.app.dexterity).
When RestrictedPython has successfully been tested against these packages porting them can start. There is a nice list of all Plone 5.1 dependencies and their status regarding Python 3.
Our goal is to complete porting RestrictedPython by the end of March 2017. It opens up the possibility guiding Zope into the Python 3 wonderland by the end of 2017. This is ambitious, especially if the work is done in spare time besides the daily customer work. You can help us by either contributing PullRequests via Github or review them.
We are planning two Zope sprints in spring and autumn 2017. Furthermore we are grateful for each and every kind of support.
After reanimation we started defibrillation of Zope and … it kinda worked:
On our sprint we got the following things done to help Zope in the Python 3 wonderland:
As grok builds on the ZTK, it is a beneficiary of the reanimation. The following steps have been undertaken to lead it to the Python 3 wonderland:
We have had a discussion about the broader future of Zope:
Conclusion: Zope is not dead. On the sprint there were nearly 20 people who use Zope for their daily work. Some of them even joined the sprint without the backup of a company in their spare time. Yes, it will need time and effort to keep Zope alive and make it prosper in the Python 3 wonderland, but Zope is still needed and has its place in the audience of web frameworks.
We at gocept will keep Zope as part of our supported technology stack in the projects it fits the purpose and will offer help to others who need to migrate a long term project into the future. We will be at the PyConDE in Munich at the end of October and will be open for questions and further discussions. Do not hesitate to talk to us.
Now we are helping Zope in the Python 3 wonderland: Almost 20 people started with the reanimation of Zope. We are working mostly on porting important dependencies of Zope to Python 3:
Zope 4 is now per default based on WSGI. Thanks to Hanno who invested much time to make the WSGI story of Zope much more streamlined.
We found out that the ZTK (Zope Toolkit) is no longer used in any of the projects using packages of it (Zope, Grok, Bluebream – as it is dead). It should be kept to test compatibility between the packages inside the ZTK.
As the Zope Resurrection sprint is approaching, it seems useful to share some information on the schedule for the three days in Halle. As we have also some sprinters, who can not join on site, but might want to join remotely, a few key facts might come in handy.
There is the Etherpad, where the current list of topics is collected. Most of the stories we are going to tackle will be based on this list. In case you have something to add, or are interested in a specific topic in particular, it is a good idea to add your thoughts before the start of the sprint.
During the sprint we will communicate via the #sprint channel on irc.freenode.net, so additional information and questions can be placed there.
As the sprint is also intended to foster discussions about the future of Zope, we want to encourage as many people as possible to join. Therefore, we have a hangout where we will meet.
At the moment we plan the following session:
So in case you want to contribute remotely to the sprint, please join us on one of the three ways.
