Viewing scales metrics from Pyramid

We’ve recently started experimenting with the excellent scales library to collect in-process metrics (see Coda Hale’s CodeConf talk “Metrics everywhere” among many others for reasons why one definitely wants to do that).

Scales comes with a flask-based HTTP server that allows viewing the collected measurements and dumping them as JSON. But if you already are in a web application, there’s no real need to spin up yet another thread, open another port etc. to do this. In our case, we’re using Pyramid, so here’s a quick recipe to get the same view that greplin.scales.flaskhandler provides:

Update 2013-11-06: This code is now released as pyramid_scales.

# in your Pyramid setup
config.add_route('scales', '/scales/*prefix')

from StringIO import StringIO
from pyramid.view import view_config
import greplin.scales
import greplin.scales.formats

@view_config(route_name='scales', renderer='string')
def scales_stats(request):
    parts = request.matchdict.get('prefix')
    path = '/'.join(parts)
    stats = greplin.scales.util.lookup(greplin.scales.getStats(), parts)

    output = StringIO()
    outputFormat = request.params.get('format', 'html')
    query = request.params.get('query', None)
    if outputFormat == 'json':
        request.response.content_type = 'application/json'
        greplin.scales.formats.jsonFormat(output, stats, query)
    elif outputFormat == 'prettyjson':
        request.response.content_type = 'application/json'
        greplin.scales.formats.jsonFormat(output, stats, query, pretty=True)
        request.response.content_type = 'text/html'
        # XXX Dear pyramid.renderers.string_renderer_factory,
        # you can't be serious
        request.response.default_content_type = 'not-text/html'
        greplin.scales.formats.htmlHeader(output, '/' + path, __name__, query)
        greplin.scales.formats.htmlFormat(output, tuple(parts), stats, query)

    return output.getvalue()

developer & admin BBQ IV

Our fourth BBQ (invitation post) had the most participants so far, almost 20 people were here to talk shop, exchange ideas and brave the unfortunately slightly rainy weather (the grilled goods were delicious regardless). We’re especially glad that the ratio of gocept people to guests was only about 50% this time, and we’re hoping it will go down further.🙂

The sessions in the Open Space were about diverse subjects, ranging from “Deploying lots of Rasperry Pi’s” over “Gamification in a business context” to “Why is there no slim and simple CMS yet?”. In several sessions we didn’t find a satisfactory solution to the problem, but sometimes sharing your frustrations with others who have similar experiences is helpful in itself.

Since the session about code katas was very well liked, we’re thinking about maybe doing a Code Retreat instead of a classic Open Space for the next BBQ, so stay tuned.

Upcoming talks at PyCon DE 2012

Most of gocept is headed towards the German PyCon next week! We’re excited and look forward to meeting people, exchanging ideas and learning new things.

We have prepared several talks to share some of the experiences we gained over the last year:

See you in Leipzig!

Autumn Developer BBQ

Our second meet/talk/barbecue event was last week, and we think it was even better than the first one.

In the unconference part, these were the topics we talked about:

  • Demo of batou, the deployment tool
  • Django
  • Sentry: centralized error-logging
  • DRBD vs. NFS vs. Sheepdog
  • Beautiful Javascript

Thanks to everyone who joined us, we’re looking forward to meeting again in December! (We’ll probably have punch instead of barbecue then…)

Our first developer BBQ

We invited developers and sysadmins to join us for talking shop and barbecuing last Friday. Even though several people had signed up and said they wanted to come, at first none of the guests seemed to arrive. But after an hour and braving some ugly traffic jams on the way here, a few did make it. We’re happy you came to visit us, guys!

In the unconference part, these were the topics we talked about:

We had a good time (and the BBQ was tasty), so we’ll definitely want to do something like this again; here’s to hoping some more people will join us next time!

Profiling class-based views

Just a quick note for profiling e.g. Zope views:

class MyView(object):
    def __call__(self):
        result = {}
        cProfile.runctx('result["x"] = super(Body, self).__call__()',
                        globals(), locals(), '/tmp/viewprofile')
        return result['x']

Even though “exec ‘result = super(…) in globals(), locals()’ works, it seems that cProfile does something a little differently here, so that writing to a local variable is not possible.

Sprint fruits: gocept.exttest and gocept.package

The whole company spent three days in Kloster Drübeck sprinting on internal tools and topics.

We overhauled our workflow for generating invoices and identified steps that we could automate.

We polished and released gocept.exttest, which integrates for example JavaScript unittest to Python’s unittest framework. In a nutshell, it allows you to write JS tests like this:

require 'my_app.js'

describe 'MyApp', ->
  it 'has read Douglas Adams', ->
  expect(new MyApp().calculate_the_answer()).toEqual(42)

and include them into your Python test suite with a single command:

import gocept.exttest
def test_suite():
    return gocept.exttest.makeSuite(
        pkg_resources.resource_filename('your.package', 'tests'))

The third area of our efforts was documentation, we designed a Sphinx skeleton to make it easy to get started writing docs, and created a template for eggs that contains the necessary boilerplate and codifies our packaging and documentation conventions. While the concrete details are probably a bit specific to our tastes, some of the general mechanincs might be interesting to others, so we’ll release gocept.package once we’ve got the missing integration tests sorted out.