At Berlin Strategic sprint 2016 we developed a tool to analyze a ZODB Filestorage to find Python 2 string objects. If they are in an encoding besides ASCII this is preventing using this Filestorage with Python 3 because of decoding errors arising on loading the pickles.
The tool is even able to convert those strings either to unicode by decoding them using a configurable encoding or convert them to
zodbpickle.binary so Python 3 will read them as
There is documentation of the tool and a repository on GitHub where the code lives.
There are still some questions open:
- Is there already another tool for this analysis/migration?
- Is there already any practical knowledge migrating Filestorage contents to Python 3?
- Do you think such a tool is the right approach to achieve such a migration?
- Is there anyone who wants to try out the tool on a Filestorage of a personal project and share the experiences? (We analyzed two projects where we have access to a Filestorage but we are sure this does not catch all the edge cases.)
We have many test suites which use test layers (e. g. the ones from plone.testing). We want to use py.test and all its fancy features to have a modern test runner. There was no way to convert such tests partly: either you have to port the whole project or you are stuck with the zope.testrunner.
On our Pyramid-Sprint Godefroid Chapelle, Thomas Lotze and me wrote a package which wraps layers as py.test fixtures. The result is gocept.pytestlayer.
For each layer it creates two fixtures: one for the layer setUp/tearDown and one for the testSetUp/testTearDown. The layer fixture is configured for class scope but the plug-in orders the tests and knows about the next test so the layer is only torn down if the next test needs another fixture.
You only have to add a new section to your package buildout and running the test via
detects the layers and displays the needed setup code. See the PyPI-Page of the package for details.
Maybe it is possible to get rid of the fixture setup code, so running tests using layers gets even easier.
Travis-CI is a free hosted continuous integration platform for the open source community. It has a good integration with Github, so each push to a project runs the tests of the project.
gocept.selenium is a python package our company has developed as a test-friendly Python API for Selenium which allows to run tests in a browser.
Travis-CI uses YML-Files to configure the test run. I found only little documentation how to run Selenium tests on Travis-CI. But it is straight forward. The following YML file I took from a personal project of mine. (I simplified it a bit for this blog post.):
- "export DISPLAY=:99.0"
- "sh -e /etc/init.d/xvfb start"
- "wget http://selenium.googlecode.com/files/selenium-server-standalone-2.31.0.jar"
- "java -jar selenium-server-standalone-2.31.0.jar &"
- "export GOCEPT_SELENIUM_BROWSER='*firefox'"
- python bootstrap.py
- Lines 1 – 4: My project is a Python project which currently only runs on Python 2.6. But other Python versions will work as well.
- Lines 5, 6: Firefox needs a running XServer, so we start it first as it takes some seconds to launch. See Travis-CI documentation, too.
- Lines 7, 8: The Selenium server seems not to be installed by default, so get it and launch it.
- Line 9: Tell gocept.selenium to use Firefox to run the tests. (Note: To use the new Webdriver-API in the upcoming version 2 of gocept.selenium you have to set other environment variables.)
- Lines 10 – 14: Install the project and run the tests as usual. (The example uses zc.buildout to do this.)
Note: Although I use the Firefox which is installed by default on the Travis-CI machine, I did not yet find out which version it is.
In a new Pyramid project we used deform to render forms. We did not really like it. (The reasons might be detailed in another post.)
To see if other form libraries do better I gave yafowil a try at our gocept Developer Punsch 3: yafowil comes with written documentation. To get a form in our Pyramid application I had to find out some things which are not so clearly documented:
- Let the project depend on yafowil.webob via setup.py as it contains the necessary WebOb integration.
- Import the
loader from yafowil like below, to allow yafowil to register all its known components (even all the packages in the yafowil.wiget namespace). Otherwise I got strange errors. (The loader symbol is not needed at all in the rest of the code of the form.)
from yafowil import loader
- To get a value displayed in the rendered form use
value keyword parameter in the factory like this:
form['name'] = factory('field:label:text',
value can be a plain value or a function which gets the widget and the runtime data of the widget as parameters.
yafowil.base.Factory.resources_for could be a starting point. (I did not do this yet, so it might be wrong.)
Conclusion: yafowil looks like an interesting framework and after getting a starting point it should be useable in Pyramid, too. Maybe this post can help to ease it a bit.
zope.formlib has the ability to customize the used widget like this:
form_fields = zope.formlib.form.Fields(IKeywords)
form_fields['keywords'].custom_widget = KWSelectWidgetFactory
I do not like this approach for two reasons:
- the widget has to be set manually every time the specific field is used
- there is no easy way to get a display widget if the form or field is not editable for the user
Defining a new schema field and registering the widget for this field seems a bit heavy, so I came up with providing a marker interface on the field:
"""Marker interface to get a special keywords widget."""
keywords = zope.schema.List(
title = _("Edit Keywords"),
value_type = zope.schema.Choice(
I registered the edit widget and display widget for the
IHaveSelectableKeywords interface, so the custom widget does not have to be set in the form like this (edit widget):