Manage JavaScript dependencies with Fanstatic

Until the beginning of this year, we were using Fanstatic to manage dependencies to external JavaScript libraries. In case you are not familiar with Fanstatic, here is a short overview. I will discuss benefits and drawbacks later on.

How it works

Imagine you want to use jQuery in one of your projects. Therefore you build an integration package, say js.jquery, which contains the jquery.js file. You also add a resource.py which would look like this:

from fanstatic import Library, Resource

library = Library('jquery', 'resources')
jquery = Resource(library, 'jquery.js', minified='jquery.min.js')

To finish up, you state the version of the downloaded jquery.js in the setup.py and upload the package to PyPI.

After wrapping your app with Fanstatic, you can declare JavaScript dependencies by listing the integration package inside your setup.py. To use them, you write js.jquery.need() inside any view. Of course you can also add a resource.py and declare a local resource with dependencies:

import fanstatic
import js.jquery

library = fanstatic.Library('custom', 'resources')
my_awesome_js_code = fanstatic.Resource(
    library, 'js/my_awesome_js_code.js',
    minified='js/my_awesome_js_code.min.js',
    depends=[js.jquery.jquery])

Every time you call my_awesome_js_code.need() inside a view, it will also load jQuery.

Benefits

With Fanstatic you no longer need to copy the JavaScript files inside your project and link them statically, you just declare them as a dependency to your own JavaScript code and call my_awesome_js_code.need() inside a view.

The preparation described above can be tedious, but the assumption is, that someone else has built the integration package before, so you can reuse it. Installing a dependency therefore is as easy as adding a line to your setup.py and declare it as a dependency of one of your resources.

Fanstatic makes it a lot easier to update external libraries, since you only need to increase the version number in your setup.py. You can also make sure to use the same version across many projects, e.g. by sharing some buildout configuration across projects.

And besides many small nice features, Fanstatic can also create bundles, i.e. it will merge my_awesome_js_code and all of its dependencies into one big file and deliver it to the client, rather delivering all files separately. This usually makes the initial page load much faster.

Drawbacks

Despite all the benefits Fanstatic offers, we were getting more and more frustrated with it. The main reason is, that we feel a decline in the activity of the Fanstatic user group. With fewer people using Fanstatic, we often have to build the integration packages ourselves. This eliminates one of the main benefits, i.e. that we could reuse the integration packages of others.

For example, in a big project we did last year, we used integration packages for js.jquery, js.classy, js.chosen and many more. Those are pretty popular packages and the integration worked out of the box.

However, even those packages have issues: The integration package for jQuery does not offer version 2.0 or higher and Chosen is only available in a single (outdated) version. We often stumbled upon similar issues with other libraries, e.g. the integration package for js.modernizr is more than 2 years behind.

In addition, some newly developed libraries may not be available at all. Of course we could write the integration packages for those or update the jQuery integration package. But this would mean to find out where the code for an integration package is hosted, write a pull request and wait for a merge, as well as a new release on PyPI. In case the owner is not active anymore, you are out of luck and must add a duplicate package to PyPI. Tedious and unsatisfactory.

This is why we started to look for an alternative to Fanstatic. And maybe we have found it, but this is a story for another day, where I will talk about our experiences with Bower and Bowerstatic.

Posted in en | Tagged | Leave a comment

Making CI results visible at Home

We are using a Jenkins server to test our projects continuously. To make the results clearly visible, my colleagues installed a Raspberry Pi to display the testing results on a huge LED strip. As you see in the picture below, the project results are displayed on both sides of the whiteboard. The aggregated result is displayed in the big LED tube on the top.

Picture of our Whiteboard with LEDs on either side displaying test results

This setup was built in 2012, but this year we started to work remotely, i.e. from home. Therefore we have no access to informations which are physically displayed at the office. Some month ago, Daniel Havlik mouthed the idea to build an LED strip “to go”. Since the decreased visibility annoyed me, I was a strong advocate of his idea.

Finally, during the Summer Sprint we held in September, we had the opportunity to work on a project of our choice. Daniel Havlik supplied us with an Arduino Uno and a small LED strip, so we started to work on his idea together with a friend of mine, Oliver Zscheyge.

During the first day, we built a simplistic API for the Arduino to set the color of an LED by providing position and color. To test the API we wrote a small Python script that sent random colors to all LEDs.

At the second day of the sprint, Daniel astonished us, since he had put in some extra hours during the night to build a prototype. It had a wooden frame, colored in black and all. Wow! This was definitely the most “visible” result of the Summer Sprint.

Prototype in a wooden case with LEDs and a USB connector

Later that day, we extended the Python script to retrieve the project states from Jenkins and set the LEDs accordingly. We used an INI-file to define which LED represents which project. For fun we also played a short sound when a project was working and broke or the other way around.

Since one of my co-workers wanted to move back to his home town, he got the prototype. To speak truly, I was a little bit jealous about that. But to my surprise, Daniel built another “LED to go” and gave it to me in October, as a birthday present. Since I am a LARPer, I love the shape of it which resembles a sword.

Remote LED with a Sword-like shape

At some days I like to pronounce that “my sword of Jenkins power is glowing green”, which casts a smile on the faces of my co-workers. I think this was a great project with a very useful result.

If you possess some crafting abilities, you can easily make an “LED to go” yourself. We made our code public at bitbucket/remoteled. It contains the versatile API for the Arduino, as well as the Python script which retrieves data from a Jenkins server. Of course you can build additional clients for your personal needs. If you think it’s useful to others as well, send us a pull request. We are looking forward to it!

With these words I am wishing you all a Merry Christmas and a Happy New Year, since I will not write another post in 2014. See you in January!

Posted in en | Tagged , , | Leave a comment

Florian Pilz

Today’s blogpost is all about me. I joined gocept about a year ago, therefore I want to tell you something about the past – and something about the future.

The Past

Florian Pilz

Before joining the gocept crew, I studied computing at the HTWK in Leipzig for 6 years. Since sitting in a german university was not enough for me, I reached out to study in the UK for the best time of a year. This way I came to the University of Bolton near Manchester, where – most importantly – I learned to speak English fluently.

During my time in the UK I started to look into Ruby on Rails and earned my first salaries. This was the point were I switched my Linux PC for a Mac. Since I already was comfortable with Windows and Linux I wanted to try out the last big operation system out there. But as many before me, I liked the Mac so much that I still use it as my development machine.

Back in Germany I continued to earn some money on the side working with Ruby on Rails. I really enjoyed using Ruby and I think in terms of readability it’s ahead of most other programming languages by several magnitudes. But Ruby also has its shortcomings, which I did not recognize until I switched to Python as my main programming language here at gocept.

I also did some very interesting projects during my studies, here are just three of them:

  • An OCR software which recognizes mathematical symbols. The idea was to integrate it into the Leibniz-project to make it possible for blind people to read mathematical school books.
  • An iPhone app that helps developers and freelancers to get better at estimating how long it takes to implement a feature or project.
  • A scientific article about software estimation that was actually published in print!

After I finished my master thesis, I had the opportunity to go on a 2-month vacation with my wife. So we took two backpacks and travelled around the entire US. A memory I will feast on my entire life. You can get some impressions on my flickr fotostream.

The Future

During the last year at gocept, I dove into the code base of two of our biggest clients – DGB and Ver.di – and helped improving their CMS. Currently I am getting in touch with the code base of Zeit Online, since a colleague will move to the upper floor, where our admins are situated.

But most importantly, you will see me more often around here, since I will be writing about things we do and learn here at gocept during the following months. I already have some topics on my desk which are worth a blogpost. For example, why hg rebase can be harmful, why testing AJAX using Jasmine is a pain, the benefits of a headless browser like PhantomJS and many more.

Next up will be a post about something we did at the summer sprint, which includes an Arduino, an LED strip and a failing test server.

Posted in en | Tagged | Leave a comment

Sommerfest bei gocept – Samstag 20. September 2014 ab 16 Uhr

english version below

Kein Sommer geht zu Ende ohne ein Fest in unserem tollen Garten. Wie in den letzten Jahren laden wir unsere Familien, Freunde und Mitstreiter ein. Für leckeres Essen, kühle Getränke und Unterhaltung ist gesorgt – kommt und feiert mit uns – wir bitten um Voranmeldung.

WANN: Samstag, 20. September ab 16 Uhr

WO: Forsterstr. 29, 06112 Halle (Saale)

KANN es nicht erwarten: mail@gocept.com

Übrigens, auch dieses Jahr bildet die Party einen Schlusspunkt zu unserem DevOps Sprint. Hier könnt ihr mehr darüber lesen: gocept DevOps Sprint

 

gocept summer party– Saturday, September 20th  4 pm

No summer ending without a party at the gocept garden. Like in previous years we invite our families, friends, and “brothers-in-arms”. As usual we serve delicious food, chilled drinks, and amusement – so come and join us – R.s.v.p.

WHEN: Saturday, September 20th from 4 pm

WHERE: Forsterstr. 29, 06112 Halle (Saale)

CAN’T wait for it: mail@gocept.com

Incidentally, the party is also the closing event for our DevOps sprint.

Posted in de, en | Tagged | Leave a comment

haproxy load-balancing for PHP applications with sticky sessions

We like applications that are written with a shared-nothing approach: it greatly simplifies running multiple instances on multiple hosts and allows for simple, robust load-balancer configuration.

Recently, we had to deploy a PHP application that – in the last minute – turned out to use PHP sessions and thus required sticky sessions.

We haven’t used sticky sessions in a while and the amount of reading required to find the specific working setup was substantial, so we’ll repeat here what a post at networkinghowtos.com already figured out:

backend default
    appsession PHPSESSID len 64 timeout 3h request-learn prefix

As you can see there isn’t much magic to it – the haproxy manual has a good in-detail explanation of the appsession option. The biggest point of this option is that you do not have haproxy injecting another session identifier but simply piggybacks on the existing one that PHP determines. Also, this option combines nicely with “leastconn” balancing if your application only uses cookies on a few selected pages and many users do not trigger getting a session cookie.

Posted in Uncategorized | 2 Comments

September, 18th–20th: DevOps Sprint

Since we have a strong history in web development, but also were involved in operating web applications we developed, the DevOps movement hit our nerves.

Under the brand name “Flying Circus” we are establishing a platform respecting the DevOps principles.

A large portion of our day-to-day work is dedicated to DevOps related topics. We like to collaborate by sharing ideas and work on tools we all need to make operations and development of web applications a smooth experience. A guiding question: how can we improve the operability of web applications?

A large field of sprintable topics comes to our mind:

Logging

Enable web application developers to integrate logging mechanisms into their apps easily. By using modern tools like Logstash for collecting and analyzing of the data, operators are able to find causes performance or other problems efficiently.

Live-Debugging and Monitoring

Monitoring is a must when operation software. At least for some people (including ourselves), Nagios is not the best fit for DevOps teams.

Deploying

We always wanted to have reproducable automated deployments. Coming from the Zope world, started with zc.buildout, we developed our own deployment tool batou. More recently upcoming projects, such as ansible, and tools (more or less) bound to cloud services like heroku.

Backup

After using bacula for a while, we started to work on backy, which aims to work directly on volume files of virtual machines.

and more…

Join us to work on these things and help to make DevOps better! The sprint will take place at our office, Forsterstraße 29, Halle (Saale), Germany. On September, 20th we will have a great party in the evening.

If you want to attend, please sign up on http://www.meetup.com/DevOps-Sprint/events/191582682/.

 

Accomodation

For your stay in Halle, we can recommend the following Hotels: “City Hotel am Wasserturm”, “Dorint Hotel Charlottenhof”, “Dormero Hotel Rotes Ross”. For those on budget, there is the youth hostel Halle (http://halle.djh-sachsen-anhalt.de/). Everything is in walking distance from our office.

Posted in en, Uncategorized | Tagged , , , , , , | 2 Comments

Flying Circus at EuroPython 2014

If you’re attending EP14, be sure to visit our Flying Circus booth at BCC level A! We’re here to discuss web operations. Managed hosting is only as good as the people behind it. So just walk over, test us, ask any question related to web operations! Additionally, we have some demo VMs readily available so you can get hands-on experience with a walk-through from our developers.

Flying Circus boot at EP14

Posted in en | Tagged , | Leave a comment