Darren Mothersele

Software Developer

Warning: You are viewing old, legacy content. Kept for posterity. Information is out of date. Code samples probably don't work. My opinions have probably changed. Browse at your own risk.

My Review of Drupal Camp London 2013

Mar 5, 2013


This last weekend was the hugely successful (and first ever!) Drupal Camp London. We had the Drupal Con in London a couple of years ago, and that was an amazing event, but somehow I enjoyed this even more. I think it was a much more manageable size, smoothly executed, and well focused. A big congratulations to the organisers. I've written up my notes from the weekend, with a bit from memory where I lost some of Saturday thanks to Evernote's security breach.

Business Day

The first day was the "business day". I've been working some long hours recently so I treated myself to a lie in by not setting the alarm. I thought I'd still make it on time, but unfortunately I missed the first half of Mark Taylor's keynote on Open Source in Government and Business. I'd seen most of this presentation before at a previous Drupal UK event. One takeaway from his presentation was that government/public sector doesn't care about licensing debates (e.g. GPL vs BSD), and it's all about cost saving. Oh, that and register on G Cloud.

Next up was the bit I couldn't miss, it's MTV. Paul Reeves gave a potted history of the use of Drupal in MTV, from being an early adopter and the challenges that bought, to the current process of standardising on Drupal and the many advantages that brings (standardisation of process). It was really interesting for me, as I'm working on the project, to see our work in context. It also generated a lot of interest in all the cool stuff we've been building.

I hovered around for the rest of the day catching up with people I'd not seen for a while, catching interesting bits of the case studies being presented. Tom Phethean from CapGemini, Joe Baker from Oxfam, Rob Collyer from Janet UK, all presented a strong case for the use of Drupal and in many cases delving straight into weak points and topics that are often used as arguments against using Drupal. Scalability, performance, security, and development workflows were all covered.

The day finished with Jeffrey (Jam) McGuire from Acquia. I'd heard from a friend who had been at TFM&A that is is possible for Acquia to do a presentation that's not an out-and-out sales pitch, but it was nice to witness it myself. Almost in direct opposing position to what Mark Taylor had said at the beginning of the day about it all being about the bottom line, Jam made a lot of sense in the way he presented the idea of being "better not cheaper" and "competing on features, not price". In comparing Drupal with a closed source competitor, he commented that the implementation costs were roughly the same, the hosting and support costs are roughly the same, but you save the licensing costs and so can spend more on the stuff you care about (design and functionality?). This is a nice match with what I love about Drupal - it's not removing the need for a web developer, it's freeing them to work on the cool stuff.

Jam also discussed an article from the Guardian about security in open source software (this one?) - a point covered several times during the day, but worth repeating - open source is not less secure than closed source. He also tackled the (sometimes tricky) job of persuading clients to contribute back, saying it's like "buying it once and getting all the upgrade for free, forever". There was also an interesting metaphor involving a restaurant, where in the restaurant business it's important to own the bricks, as when you're successful, and if you lease the premises, landlords can increase the rent.

And, that was just the business day! And, there was even more to it than that. Lots of conversations were going on around the place about Drupal 8, what's in, what's out. What should have been in, what shouldn't be in. Content staging, inline editing, addressing the needs of enterprise clients.

Drupal Camp Day One

So, I missed the keynote again. Probably attributable to the massive hangover after the Drupal Camp social the night before. Somehow managed to miss my stop on the tube, and so decided to take a long walk to the venue instead to blow away the cobwebs.

I arrived just as David Axmark (one of the co-founders of MySQL) began to talk about SQL vs NoSQL. As you can imagine, he doesn't have an unbiased view on this, but he did make some points about SQL systems being well understood and developed over about 40 years, and even now starting to support dynamic columns. Oh, and he made a point about performance optimisation by using direct row access, but that sounded scary.

First session of the day, I chose Drush Make Driven Development with Steven Jones from Computerminds. This was a nice easy start for my hungover brain as I've been doing this for a long while anyway, but I did hope to pick up some new tricks, and I did - it's called Drush Remake. Thanks Steven.

Next up was "Code Driven Development" Features and Beyond, with Steven Richards. I went along to this session mainly for the "Beyond" bit - of which there didn't seem to be any. I wish people would stop calling Features code driven development. It's not. But that's another subject, for another post some time in the near future.

Then after a bit more hungover bumbling I found Steve Purkiss' Business of Community session. I'd just recently registered with the Drupal Coop, and was interested to hear what he had to say. Instead, we were treated to a really interesting film (now available online), seriously, go watch it.

After this I found myself in a presentation about the Scald module. A competitor to the Media module. This was just a good example of what the Drupal community SHOULD NOT do. Yes, the media module is not perfect, and Scald looks great, has a nice UI, but I left the session thinking how awesome it would be if there was some combination of media and scald, combining the best of both. A good demonstration of why we should be collaborating, not competing.

I'm starting to shed the hangover now, so I make my way to Greg Harvey's "Make Your Cheap VM Fly". The anti-climax is that Greg's answer is mainly just "installed Nginx". But, saying that, I did come away with some useful tips of things to try: "Percona, it's like mysql but quicker, like mariadb but less of a departure from mysql", "Drupal APC needs 128MB shm_size to be effective", "Varnish does more harm than good if you've got less than 4GB", "memcache works well with about 512MB".

I've been doing loads of migration stuff recently, so I was excited about Ed Crompton's "Testing for Large Scale Data Migration". He presented an interesting idea, and one that I had not considered, which is to use WebTestCase (part of Drupal's SimpleTest), to test content that had been migrated. The idea is that you extend the WebTestCase class to stop it cloning the DB, so you can run tests against the current DB - scary! The base MigrationTestCase that was presented also did some other useful stuff, like loading up source and destination rows, and providing logging to CSV. This all sounded fun, but I was unsure of the point. I'd achieved exactly the same by logging exceptions as part of the migration process. There may be benefits from writing your tests separately to your migrate code, but is it worth dealing with slooooooow SimpleTest?

To end the day, I went along to Marcus Deglos' Vagrant crash course. I've heard a lot about this tech over the past few months, and I've been meaning to check it out. I already use separate VirtualBox environments for my local dev, so I'd already had to solve some of the issues (like using Samba to get around permissions problems). I'd seen a video recorded at a previous Drupal Camp somewhere else in the world (I forget where) which had left me thinking it was going to be more difficult than this, but Marcus made it very clear. He's even provided his Vagrant scripts on GitHub to make it even easier.


I made it in on time for the Keynote! Yay. I definitely didn't want to miss this, as it was about Drupal distributions. I wasn't sure what was going on at first. Robert started in monotone, and I started getting this strange sense of deja vu. Then I realised he was reading something, a blog post by Dries perhaps? I was right, but I didn't realise how old the post was until he said. It was Dries' post from 2006 about distribution support in D5. The promise of Drupal Distributions has been slowly burning (very slowly) since then, and only now has it really started to take hold. But, as Robert explained, we can do much better.

Robert used the image gallery example consistently throughout his presentation. Starting with the original image module, we had an "image gallery" feature in Drupal, but over time we abstracted it and we're left with Views, Image fields, etc. Drupal has these great building blocks, but we don't have a killer image gallery. Robert demoed the slick UI of Facebook or Google+ image galleries, and then compared this to the best we have now, a nine step "recipe" for building an image gallery in Drupal. Robert tells us we have to "bring it home". And this is the promise of Drupal distributions, and Apps.

What's an App? This is something I looked at briefly, back when I used to think features was the answer. But, then Apps were very new, and not really being used. Now they are starting to appear in the main Drupal distributions, and Commerce Guys are even launching an App Store for their Commerce Kickstart distribution. I'd already been discussing Apps with Kristof Van Tomme on Friday evening, and I'd started to realise I probably want to build an App, not a distro (more about that in a future blog post!). I'd been put off by the use of Features before, but I'm starting to look at it again. There are lots of technical hurdles to overcome, such as managing shared configuration. If you've tried to do anything with Features you'll have come across this problem when you start getting conflicting features. This isn't such an issue with "distros with a common lineage", as Robert puts it. I.e. The distros from the same vendor, such as the OpenPublish, OpenAtrium, OpenPublic distros from Phase2. http://www.phase2technology.com/

But, the biggest hurdle we have to get across, is the concept of paid for apps and commercial modules. Unlike Wordpress, that has a thriving marketplace for commercial themes and paid for extensions, this has been a very controversial topic in the Drupal community. This conversation continued later in the day in a BoF...

A quick coffee, and a chat with George Boobyer about the question I'd asked during the keynote. Yeah, I'm not using Features, and I haven't for some time, but sometimes I feel like the only person who's given up on it. It's not really code driven development. There is a better way. I'll blog more about this, as clearly not many people have realised yet! I think George agreed, but I very rudely ran away as I didn't want to miss the first session!


I'd used the Drupal 7 Queue API for the first time on the TrueTube website for managing video submissions and transcoding. It was only while working on the MTV project that I realised the full potential of using queues. I now want to use queues for everything. It's so liberating, and it really makes the admin area of sites snappy and responsive. No more waiting around for and edit form to submit. Offload all the processing, anything that doesn't definitely have to happen on the page request should be queued up for asynchronous processing.

Tom Phethean from Cap Gemini gave a good overview of the Queue API, and had some good tips and tricks along the way. There was some discussion of simple in memory queues, vs reliable database queues, and I realised I should probably look at the STOMP protocol at some point. One phrase that stuck in my mind was "death by log", a horrible thought, and I made a note to look at GrayLog at some point. I'm using Beanstalkd as a queue system, which got a mention, and I had to ask about demonising PHP. I'm using Supervisord, although we've not put this into production yet. Tom said he'd not tried this approach yet, and recommended lots of testing! I made a note to look at Drush queue running, rather than using the queue runner provided by the beanstalk module to see if that offered any advantages.

Next up, Jim from Code Enigma packed out a full room, probably of worried themers, as his session was titled "Death of the Themer". The basic premise was to remove the theming job and replace it with configuration. For someone who likes to do as much as possible in code, I wasn't sure this was going to work for me, but it sounds like it could still be compatible with my approach. I made some notes to try a few things, and perhaps I'll blog the results. Jim's basic premise is that the front end designers should be free to do what they do best with all the cool latest tools they like to play with, and Drupal shouldn't get in the way. There was a huge impromptu round of applause when Jim flashed up clean HTML output from Drupal. :)

The actual sessions part of the camp was nicely concluded by Ronald Ashri's session "Architecting Drupal Modules". As Michael Lenahan pointed out, it's like being back at uni. Ronald's academic background is clear in his presentation. I remember seeing a session he did on the Entity API back when Drupal 7 was new, and having a lightbulb moment, and this was another of those sessions. "It's not about the recipe, it's about the principles", Ronald tells us. We have to stop talking about problems in Drupal language, and Ronald points out that if we change the way we talk about problems, we change the way we think about them.

Ronald explains a hardcore application of MVC to building Drupal modules. In terms of the logic bits, he talks about presenting your functionality in clean OO patterns, and separating it from the "Drupally" stuff like submit handlers. "Solve your problems, then solve Drupal's problems". There were some questions about use of entities as the "model" bit of MVC, to which Ronald replies, "it's true that Drupal is not a clean model, but if you think about it in that way then you are clear of the compromises you are making."

I spent a large portion of the afternoon in a lively BoF on the subject of distributions, apps and paid for Drupal extensions. There was some talk of the technical challenges, although several attendees dismissed them as "just technical issues". I would have liked a bit more discussion on this, as I find it hard to see how an wide open market place for apps could ever work with any level of compatibility and interoperability. We would need common scaffolding in terms of the environment that the app is injecting into. Managing shared (and possibly conflicting) configuration seems like a very hard problem to solve. I had a look at nedjo's work on app compatibility, and it is excellent work, but we'll have to agree on lots of standards and it's much more than just shared taxonomy and user roles. That would be a great start, but there's a lot more to it than this.

Anyway, the BoF focused instead on the commercial aspect, and I agree this is the biggest hurdle, as it requires a cultural change in the Drupal community. Robert told how there was a starting to be a shift in opinion as he recalled a previous session a few years earlier where the idea was met with a lot of resistance. Robert picked up on something that Ronald Ashri said, and commented "notice the shift in audience, selling to travel agents, not site builders".

Several of the BoFers said they would be happy to pay for smooth execution of a feature. Only one person at the BoF had a strong objection to commercial modules for Drupal. Robert gave an example of a wordpress plugin that was initially uploaded to the Wordpress site for free, but was eventually taken off the free market and went commercial. Rather than the backlash that was expected the guy received more thanks from users who's clients were happy about the change as before were reluctant to trust it because it wasn't commercial.

Some hypothetical scenarios discussed included what would happen if something like Views had been commercial. Would this have been an incentive to not move it into core? Suggestions were that Apps would not be building block modules, but rather well developed features for a specific use case.

It's a difficult topic and there are opportunities out there. Wordpress has a huge market for paid for extensions. Drupal looses third party app vendors because there is no market place for them to sell, so there's no incentive for them to build modules. But, Drupal attracts big enterprise projects because of the strength of it's eco system and solid foundations. Would commercial modules and app stores dilute this?