{"id":1525,"date":"2015-03-06T15:50:39","date_gmt":"2015-03-06T14:50:39","guid":{"rendered":"http:\/\/blog.gocept.com\/?p=1525"},"modified":"2015-03-06T15:50:39","modified_gmt":"2015-03-06T14:50:39","slug":"manage-javascript-dependencies-with-bowerstatic","status":"publish","type":"post","link":"https:\/\/blog.gocept.com\/2015\/03\/06\/manage-javascript-dependencies-with-bowerstatic\/","title":{"rendered":"Manage JavaScript dependencies with BowerStatic"},"content":{"rendered":"<p>Last month I explained how to use <a href=\"http:\/\/blog.gocept.com\/2015\/01\/19\/manage-javascript-dependencies-with-fanstatic\/\">Fanstatic to manage JS dependencies<\/a>. Since we were more and more displeased by using <a href=\"http:\/\/fanstatic.org\">Fanstatic<\/a>, we recently switched to <a href=\"http:\/\/bowerstatic.readthedocs.org\">BowerStatic<\/a>, the new kid on the block. Since the setup is a bit more complicated and you need more tools to have the same features as you had with Fanstatic, I will describe how to set up the full toolchain. After that, I will shortly summarize the benefits and drawbacks of BowerStatic, so you can decide for yourself if you prefer BowerStatic above Fanstatic or the other way around.<\/p>\n<p>Of course the choices made in the toolchain of BowerStatic are strongly linked to other tools we use. Since we set up large projects using <a href=\"http:\/\/www.buildout.org\">Buildout<\/a>, we like to use Buildout recipes to solve a problem. Therefore this blogpost is most interesting when you are using Buildout yourself.<\/p>\n<h2>How it works<\/h2>\n<p>BowerStatic was created by <a href=\"https:\/\/github.com\/faassen\">Martijn Faassen<\/a>, who was a core developer of Fanstatic. We think of BowerStatic as a reimagined version of Fanstatic, avoiding disadvantageos decisions made in the past. One problem of Fanstatic was, that it tried to do too many things: Fetching the files, delivering them to the client and bundling them into one large file.<\/p>\n<p>Therefore BowerStatic has a narrow focus: It only helps to manage which libraries should be delivered to the client. It does not care how the files were retrieved or whether you want to have a bundled resource. To automate these steps as well, you must look for additional tools.<\/p>\n<p>We use the following toolchain:<\/p>\n<ul>\n<li>a Buildout recipe for <a href=\"http:\/\/bower.io\/\">Bower<\/a> to download all external libraries<\/li>\n<li><a href=\"http:\/\/bowerstatic.readthedocs.org\">BowerStatic<\/a> to deliver JS files to the client<\/li>\n<li>a custom Buildout recipe to bundle all files of the project into one large file<\/li>\n<\/ul>\n<p>Since you need all of these tools to have the same features as Fanstatic, I will give a short description, how to set things up.<\/p>\n<h2>Bower<\/h2>\n<p><a href=\"http:\/\/bower.io\/\">Bower<\/a> is a small tool to download libraries and their dependencies by using a simple description of the library in JSON, which is named <code>bower.json<\/code>. The description contains a version string, the path to contained files, a list of libraries it depends on and a lot more metadata like author, homepage, keywords and so on. For example:<\/p>\n<pre>\r\n{\r\n  \"name\": \"bootstrap\",\r\n  \"version\": \"3.3.2\",\r\n  \"main\": [\r\n    \"less\/bootstrap.less\",\r\n    \"dist\/css\/bootstrap.css\",\r\n    \"dist\/js\/bootstrap.js\",\r\n    \"dist\/fonts\/glyphicons-halflings-regular.eot\",\r\n    \"dist\/fonts\/glyphicons-halflings-regular.svg\",\r\n    \"dist\/fonts\/glyphicons-halflings-regular.ttf\",\r\n    \"dist\/fonts\/glyphicons-halflings-regular.woff\"\r\n  ],\r\n  \"dependencies\": {\r\n    \"jquery\": \"&gt;= 1.9.1\"\r\n  }\r\n}\r\n<\/pre>\n<p>The great thing about Bower is, that you just point to a github repository in case this certain library is not registered on the Bower server. In this case it will check out the trunk of the repository and creates the <code>bower.json<\/code> all by itself using the commit hash as it&#8217;s version. Of course it cannot auto-detect dependencies, which are normally declared in the <code>bower.json<\/code>.<\/p>\n<p>It&#8217;s even better if the repository contains a <code>bower.json<\/code> file. In this case Bower will only load the files mentioned there and can retrieve dependencies. Therefore it makes no difference if you point Bower to a registered package, e.g. <code>jquery-ui<\/code>, or if you point it to the github repository of this project, e.g. <code>github.com\/jquery\/jquery-ui<\/code>. Of course the version will be different (stable release vs trunk).<\/p>\n<p>To download external dependencies via Bower, we added <a href=\"https:\/\/pypi.python.org\/pypi\/bowerrecipe\">bowerrecipe<\/a> to our Buildout configuration. This way we can just list the libraries and their version in the recipe configuration. Bower will automatically download those libraries, as well as their dependencies. The following example will download Bootstrap and jQuery, since Bootstrap depends on jQuery.<\/p>\n<pre>\r\n[bower]\r\nrecipe = bowerrecipe\r\npackages =\r\n    bootstrap#3.3.x\r\nexecutable = ${buildout:bin-directory}\/bower\r\n<\/pre>\n<p>By default the libraries will be stored inside <code>parts\/bower\/downloads<\/code>. The destination can be changed inside the recipe configuration.<\/p>\n<h2>BowerStatic<\/h2>\n<p><a href=\"http:\/\/bowerstatic.readthedocs.org\">BowerStatic<\/a> uses libraries downloaded via Bower to resolve JS dependencies and delivers all required files to the client. To do so, it must be registered as a WSGI middleware. Compared to Fanstatic, the setup of BowerStatic is a little bit more complicated:<\/p>\n<ol>\n<li>Create an instance of the Bower class<\/li>\n<li>Create a external component collection, which contains all external libraries loaded into the directory <code>parts\/bower<\/code><\/li>\n<li>Create a local component collection, which will contain your self-written JS files<\/li>\n<li>Set the external collection as the fallback of your local collection, i.e. JS libraries not found in the local collection will be looked up in the external collection<\/li>\n<li>Add your self-written files to the local collection, by reading a <code>bower.json<\/code> at given path<\/li>\n<\/ol>\n<p>This may sound like voodoo to you. The <a href=\"http:\/\/bowerstatic.readthedocs.org\/en\/latest\/integrating.html#integrating-bowerstatic-with-a-wsgi-app\">documentation of BowerStatic<\/a> contains some insightful statements why the setup is so complicated. Written in code the setup will look like this:<\/p>\n<pre>\r\nimport bowerstatic\r\nimport os\r\nimport pkg_resources\r\n\r\nbower = bowerstatic.Bower()\r\nexternal_components = bower.components(\r\n    name='components', path='parts\/bower\/downloads')\r\nlocal_components = bower.local_components(\r\n    name='local', component_collection=external_components)\r\n\r\nlocal_components.component(path=pkg_resources.resource_filename(\r\n    'my.cool.package', 'resources'), version=None)\r\n<\/pre>\n<p>Let&#8217;s assume in <code>my.cool.package.resources<\/code> is a <code>bower.json<\/code> like:<\/p>\n<pre>\r\n{\r\n  \"name\": \"custom\",\r\n  \"version\": \"0.1\",\r\n  \"main\": [\r\n    \"custom.js\",\r\n    \"custom.css\",\r\n  ],\r\n  \"dependencies\": {\r\n    \"bootstrap\": \"*\"\r\n  }\r\n}\r\n<\/pre>\n<p>Then you have the JS library called <code>custom<\/code> in your local component collection. After handling this setup, you can use BowerStatic similar to Fanstatic, by including JS files inside any view. However, you always need the component collection to do so. <code>local_components.include(self.request.environ, 'custom')<\/code> will load <code>custom.js<\/code>, <code>custom.css<\/code>, the bootstrap library and the jQuery library. You could also write <code>local_components.include(self.request.environ, 'jquery')<\/code>, which would only load jQuery.<\/p>\n<h2>Bundling<\/h2>\n<p>Since we were used to Fanstatics automatism to bundle and minify all resources, we wanted to have something similar with BowerStatic. It is not sure if BowerStatic will ever contain bundling mechanisms, since this may be out of the narrow scope it wants to preserve.<\/p>\n<p>Therefore we build our own Buildout recipe, <a href=\"https:\/\/pypi.python.org\/pypi\/gocept.recipe.bowerstaticbundler\">gocept.recipe.bowerstaticbundler<\/a>. This recipe will create another Bower library in <code>parts\/bower<\/code>, which contains a minified JS file, a minified CSS file and assets like images and fonts. By using the dependency mechanism of BowerStatic, all JS files are bundled in the right order.<\/p>\n<p>To use the bundled library in production and the separate files in development mode, we build a custom include with an environment variable switch:<\/p>\n<pre>\r\ndef bower_include(environ, name):\r\n    if not os.environ.get('BOWERSTATIC_DEBUG'):\r\n        name = 'bowerstatic_bundle_' + name.replace('.', '_')\r\n    include = local_components.includer(environ)\r\n    include(name)\r\n<\/pre>\n<p>Of course you must use this include method instead of using the BowerStatic component directly, i.e. replace alo occurences of <code>local_components.include(environ, name)<\/code> with <code>bower_include(environ, name)<\/code>.<\/p>\n<p>This solution is good enough for ourselves, but may not be ready for your purposes. We have build it with a certain project in mind, so you might come across bugs when using it with different JS libraries. Since we will soon use it in production, we will happily fix bugs you report and merge tested improvements. You can find the code and issue tracker <a href=\"https:\/\/bitbucket.org\/gocept\/gocept.recipe.bowerstaticbundler\">on bitbucket<\/a>.<\/p>\n<h2>Benefits<\/h2>\n<p>The combination of Bower, BowerStatic and a bundling mechanism has the same benefits as Fanstatic, i.e. that you no longer need to copy JS files into your project and link them statically, as well as the benefits of bundled resources to speed up the initial page load.<\/p>\n<p>The biggest contrast of these solutions is how external libraries have to be prepared: With Fanstatic you had to build an integration package for each library and release a version on PyPI for each version of the library. With BowerStatic it&#8217;s enough to have a current <code>bower.json<\/code> inside the library. So both need some kind of work and maintenance.<\/p>\n<p>However, with Fanstatic the assumption was that someone else already built the integration package for you, so that you had to do nothing. As described in the <a href=\"http:\/\/blog.gocept.com\/2015\/01\/19\/manage-javascript-dependencies-with-fanstatic\/\">Fanstatic article from last month<\/a>, this assumption is no longer valid and many JS libraries are not supported or out of date. Updating a foreign integration package consumes time and may not be possible at all, in case the user is no longer active.<\/p>\n<p>With BowerStatic, you can simply point to a git repository and select the version by using a certain commit or tag. As mentioned before, Bower will download the trunk of the repository and generates a simple <code>bower.json<\/code> automatically. Therefore every JS library that is hosted on github can be used with BowerStatic without additional work. This means no integration packages anymore, yay!<\/p>\n<p>The downside is that you must declare dependencies of those libraries as your dependencies, since Bower cannot detect dependencies when it automatically creates a <code>bower.json<\/code> from the trunk of the repository. However, if there is a <code>bower.json<\/code> inside the repository, Bower will use it and to resolve dependencies.<\/p>\n<p>The possibility of Bower to download a github repository comes with the benefit that it is really easy to try out different libraries for the same task. With Fanstatic you had to build an integration package for every experiment or copy the files into your resources. Now you can just point to the github repository and give it a try. Fanstatic always felt like a big hindrance when trying out new libraries.<\/p>\n<h2>Drawbacks<\/h2>\n<p>In my opinion the biggest drawback of BowerStatic is the long toolchain. You have to understand how Bower works, how BowerStatic uses Bower and how to set it up. Building a custom recipe for bundling was a bit time consuming as well.<\/p>\n<p>After about 2 weeks we had aggregated enough wisdom to use the toolchain smoothly. We think of it as an investment in the future, since we were so annoyed by Fanstatic, building one integration package after the other.<\/p>\n<p>Another big downside is, that there is no obvious solution to bundling when using BowerStatic. Of course you could bundle the resources using Grunt or any other approach of your choice. But we wanted a solution that fits into our toolchain, i.e. a solution that works with Python and integrates into Buildout. We did not know of any solutions that matched this criteria, therefore we created the Buildout recipe ourselves. However, building the recipe consumed some time and it is nowhere near &#8220;complete&#8221;. I hope that there will be a bundling solution that integrates nicely with BowerStatic in the near future. Till then, the Buildout recipe is good enough.<\/p>\n<h2>Conclusion<\/h2>\n<p>Wow, that was a long article. I hope it was interesting for you to read about BowerStatic. Chances are that you did not know about it before. Setting it up did take some time, but maybe the code snippets above help you to speed things up.<\/p>\n<p>All in all I am really glad that we switched to BowerStatic. After the initial learning investment, you know almost everything that you need to know for the future. Testing brand new JavaScript libraries is a breeze. Even using an unknown library takes no additional effort. Sounds like a win to me.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Last month I explained how to use Fanstatic to manage JS dependencies. Since we were more and more displeased by using Fanstatic, we recently switched to BowerStatic, the new kid on the block. Since the setup is a bit more complicated and you need more tools to have the same features as you had with &hellip; <a href=\"https:\/\/blog.gocept.com\/2015\/03\/06\/manage-javascript-dependencies-with-bowerstatic\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Manage JavaScript dependencies with BowerStatic&#8221;<\/span><\/a><\/p>\n","protected":false},"author":21108265,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_newsletter_tier_id":0,"footnotes":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[10221],"tags":[457],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/pFP3y-oB","jetpack-related-posts":[{"id":1510,"url":"https:\/\/blog.gocept.com\/2015\/01\/19\/manage-javascript-dependencies-with-fanstatic\/","url_meta":{"origin":1525,"position":0},"title":"Manage JavaScript dependencies with Fanstatic","author":"","date":"January 19, 2015","format":false,"excerpt":"Until the beginning of this year, we were using Fanstatic to manage dependencies to external JavaScript libraries. In case you are not familiar with Fanstatic, here is a short overview. I will discuss benefits and drawbacks later on. How it works Imagine you want to use jQuery in one of\u2026","rel":"","context":"In &quot;en&quot;","block_context":{"text":"en","link":"https:\/\/blog.gocept.com\/category\/en\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":236,"url":"https:\/\/blog.gocept.com\/2012\/10\/24\/upcoming-talks-at-pycon-de-2012\/","url_meta":{"origin":1525,"position":1},"title":"Upcoming talks at PyCon DE 2012","author":"","date":"October 24, 2012","format":false,"excerpt":"Most of gocept is headed towards the German PyCon next week! We're excited and look forward to meeting people, exchanging ideas and learning new things. We have prepared several talks to share some of the experiences we gained over the last year: Christian Kauhaus: nagiosplugin library Christian Theune: batou deployment\u2026","rel":"","context":"In &quot;en&quot;","block_context":{"text":"en","link":"https:\/\/blog.gocept.com\/category\/en\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":349,"url":"https:\/\/blog.gocept.com\/2012\/12\/17\/yafowil-in-a-pyramid-project\/","url_meta":{"origin":1525,"position":2},"title":"yafowil in a Pyramid project","author":"Michael Howitz","date":"December 17, 2012","format":false,"excerpt":"In a new Pyramid\u00a0project we used deform\u00a0to render forms. We did not really like it. (The reasons might be detailed in another post.) To see if other form libraries do better I\u00a0gave yafowil\u00a0a try at our\u00a0gocept Developer Punsch\u00a03: yafowil comes with written documentation. To get a form in our Pyramid\u2026","rel":"","context":"In &quot;en&quot;","block_context":{"text":"en","link":"https:\/\/blog.gocept.com\/category\/en\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":123,"url":"https:\/\/blog.gocept.com\/2011\/12\/06\/getting-sys-path-out-of-buildout\/","url_meta":{"origin":1525,"position":3},"title":"Getting sys.path out of buildout","author":"","date":"December 6, 2011","format":false,"excerpt":"A wrinkly part of buildout's design is that the PYTHONPATH is not easily available outside of scripts generated by buildout itself. I've been using the following workaround in some of my development tools for a while and found it quite helpful, even though it's hacky and rough around the edges:\u2026","rel":"","context":"In &quot;en&quot;","block_context":{"text":"en","link":"https:\/\/blog.gocept.com\/category\/en\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3219,"url":"https:\/\/blog.gocept.com\/2018\/04\/17\/allow-hosts-in-buildout-considered-harmful\/","url_meta":{"origin":1525,"position":4},"title":"&#8220;allow-hosts&#8221; in buildout considered harmful","author":"Michael Howitz","date":"April 17, 2018","format":false,"excerpt":"Today we had the following error message when re-installing a project from scratch: While: Installing. Getting section application. Initializing section application. Installing recipe zc.zope3recipes. Getting distribution for 'zc.zope3recipes==0.13.0'. Error: Couldn't find a distribution for 'zc.zope3recipes==0.13.0'. Yes this is a really old recipe but it still exists on PyPI. We are\u2026","rel":"","context":"In &quot;en&quot;","block_context":{"text":"en","link":"https:\/\/blog.gocept.com\/category\/en\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":268,"url":"https:\/\/blog.gocept.com\/2012\/11\/09\/python-2-and-3-compatible-builds-with-zc-buildout\/","url_meta":{"origin":1525,"position":5},"title":"Python 2 and 3 compatible builds with zc.buildout","author":"","date":"November 9, 2012","format":false,"excerpt":"Creating a single-source build environment with zc.buildout that works for both Python 2 and 3 is a bit of a hassle. This blog post shows how to do it for a minimal demo project. During the sprints at PyCon DE 2012, we tried to make the upcoming 1.0 release of\u2026","rel":"","context":"In &quot;en&quot;","block_context":{"text":"en","link":"https:\/\/blog.gocept.com\/category\/en\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/posts\/1525"}],"collection":[{"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/users\/21108265"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/comments?post=1525"}],"version-history":[{"count":12,"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/posts\/1525\/revisions"}],"predecessor-version":[{"id":1537,"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/posts\/1525\/revisions\/1537"}],"wp:attachment":[{"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/media?parent=1525"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/categories?post=1525"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.gocept.com\/wp-json\/wp\/v2\/tags?post=1525"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}