The new year is bringing some big changes for me. A few weeks back I accepted a position at Relaxed Inc. and notified Mozilla that I would be leaving at the end of the year.

Mozilla

I started working at Mozilla 2 years ago. I started the day after my employment at the Open Source Applications Foundation ended. At this point I already took for granted some of the best parts of working at Mozilla; working for a public benefit organization, spending 100% of my time working on Open Source, working with very smart people in the open (lists, IRC, etc.).

But Mozilla is even more than all that. Succeeding at Mozilla means something more than a pat on the back and a good end of the year review. When you succeed at Mozilla you impact one of the most important products on the internet. You reach hundreds of millions of users and contribute to keeping the web an open and free (as in speech) world. There is no other place in the world you can work where you can conceivably have this kind of impact.

Mozilla as an organization is truly unique. Last year was the hardest I’ve ever had, i suffered a huge loss in my personal life and Mozilla was as supportive during this time as any of my friends or family. There are a lot of places that let you put so much of yourself in to the organization to help it attain it’s goals but there are only a handful that are there to support you when you need it.

Relaxed Inc.

I started using CouchDB in 2008 after a great talk by Jan Lehnardt at OSCON. I started using it right away and over the next year it re-shaped how I think about web development and applications. In the last 6 months my group at Mozilla has become a heavy CouchDB user and not just because of my own interest but because CouchDB was the only solution for some of the harder problems we needed to solve with our results storage.

As I’ve used CouchDB more and more and become a part of the CouchDB community I’ve had the pleasure of knowing some of the core contributors, three of which have decided to found a new startup around CouchDB; Jan Lehnardt, J Chris Anderson, and the creator of CouchDB Damien Katz. Shorty after they received their funding they made an offer. It’s an amazing opportunity and while the decision to leave Mozilla is one of the hardest I’ve ever had to make I’m very excited about my future at Relaxed.

The Future

I’m really looking forward to working with everyone at Relaxed. It’s an exciting time and I’m not 100% sure yet which projects I currently work on that I will still have time to maintain. In the next week or so I’ll be doing a blog post on all the libraries I currently work on and maintain (it’s a long list) and what their status is moving forward. I still maintain code I wrote long before I worked at Mozilla and have every intention of continuing to work on some of the projects I started at Mozilla.

One thing is certain. I’m not the guy who figures out how to test the browser any more. Windmill and Mozmill are important projects that I have every intention of supporting by making time for code reviews and community support but I won’t be available to put time in to new feature work and refactoring like I have in the past. Luckily there are solid communities behind both of these projects and I’m confident that there are people who can continue to drive them in the future.

I don’t know what is going to happen next, all I know is that it should be fun, it won’t be like anything I’ve done before, and will certainly continue to include lots JavaScript and Python.

For everyone who depends on me and the code I’ve written over the last few years I’ll be sure to keep you all up to date. And one thing I can promise is that if you want to fix anything in one my projects, fork it on github and send me a pull request and I will always find time to look at it

I’m in London for the next few days and would love to grab a drink with any community members be you Mozilla, CouchDB, Python, Windmill, JavaScript or just plain old coffee, whisky or beer geeks

I’m getting all packed up and leaving Sunday for [EuroPython](http://www.europython.eu/) in Birmingham, UK.

This will be my first time at EuroPython and my first time in Europe!

I’ll be giving two talks, one on [Windmill](http://www.getwindmill.com) and one about [CouchDB](http://couchdb.apache.org/) and Python. The Windmill talk will be more or less the talk that I gave at [Open Source Bridge](http://opensourcebridge.org/) last week, which went very well. This is the first time I’ll be talking about CouchDB, the most exciting new technology on the web. The talk will mostly be about breaking our old data modeling habits that we developed to deal with SQL and what libraries and tools are available for interacting with CouchDB in Python.

I will also be in London for a few extra days after the conference so anyone interested in a meetup should ping me.

I’ll be leaving tomorrow morning for [Open Source Bridge](http://opensourcebridge.org/) in Portland, Oregon.

I’m putting together a new [Windmill talk](http://opensourcebridge.org/sessions/36) that tries to incorporate all the feedback we’ve received over the last year of speaking which I’ll be presenting on Thursday.

Mozilla is also a [sponsoring](http://opensourcebridge.org/sponsors/) the conference and there is going to be some great [Firefox related sprints in the hacker lounge](http://opensourcebridge.org/wiki/Hacker_Lounge). Dietrich is also giving what sounds like an awesome talk on extending Firefox called [Firefox Switchblade](http://opensourcebridge.org/sessions/251).

Hope to see you all there!

PS. I’ll also be at EuroPython and the Community Leadership Summit, more on those later

So much good stuff landed in Windmill over the last few weeks that we decided to push another major release.

The biggest new features are:

* [django management command](http://trac.getwindmill.com/wiki/WindmillAndDjango) for running windmill tests ([Jacob](http://jacobian.org/) said the existing django support wasn’t good enough and I agreed so I wrote this during the PyCon Sprints)
* new [nose plugin](http://trac.getwindmill.com/wiki/BookChapter-5-RunningTests#RunningTestsfromNose)
* cygwin support contributed by [Simon Law](http://sfllaw.livejournal.com/) (he went and wrote an implemenation of [winreg for cygwin](http://pypi.python.org/pypi/cygwinreg) to get this to work).

There were also some really good bug fixes that landed:

* much better unicode handling and serialization (adam)
* fix for POST to foreign domains ([Anthony Lenton](http://anthony.lenton.com.ar))
* continued improvements to click simulation (adam)

The release is [up on PyPI](http://cheeseshop.python.org/pypi/windmill) and you can install/update with:

$ easy_install -U windmill

For anyone interesting in working **on** windmill we’re having a Sprint in #windmill on irc.freenode.net tomorrow April 8th, 2008 for pretty much all day. We’re going to be improving the unittests for Windmill itself.

The next planned major release will be 1.2 which will include the much anticipated SSL support, courtesy of some great work being done by Anthony.

The video from [PyCon of the Functional Test Tools Panel](http://pycon.blip.tv/file/1947342/) has been making the rounds and one comment in particular that seems to be gaining some traction is a comment [Jason Huggins](http://twitter.com/jhuggins) made about test recorders being “evil”. I didn’t comment about it while I was on the panel because I have mixed experiences with recorders in the tools I work on and some of those experiences have left me feeling the same way.

I agree with Jason that recorders that generate code with the expectation that you will be able to **just run** that code as a test are definitely *evil*. Aside from just **not working** most of the time they create false expectations about test tools. But, I disagree that they can’t be implemented in a way that is useful and side steps a lot of those false expectations. **Note: On twitter Jason has added that he agrees with my disagreement probably for the same reasons I’ll highlight below.**

Looking at two tools I’m still an active developer on, [Windmill](http://www.getwindmill.com) and [mozmill](http://code.google.com/p/mozmill), both have a recorder but one I find highly successful and the other I’ve struggled greatly with.

The recorder in the Windmill IDE is phenomenal and almost entirely due to the work that [Adam](http://www.adamchristian.com/) has put in to it. The entire test authoring UI in Windmill is *action* based meaning there is a series of interface simulations you can do with a single variable (an element lookup or in some cases a string to eval). Each action is a cell that can be moved around and new actions added above or below. The same UI is leveraged for running and debugging tests, each cell shows green or red if a test has passed or failed.

The reason I like this UI and similarly simple UI’s like [FireUnit's UI](http://www.mikealrogers.com/archives/327) is that they leverage the relative simplicity of the test APIs. There is no code in the Windmill IDE, just actions, so when you record a set of actions they don’t show you code just the actions **as you interact with the page**.

Now look at the mozmill UI. Mozmill suffers from some additional necessary complexity. In mozmill we have to test multiple windows, which requires a more complex API and some required setup steps. Mozmill can’t have the same action” based UI because we have two additional variables to every UI interaction, the window (which usually requires an additional API to pull out) and the document (also requiring an additional API for easy access). Removing the complexity of the API from the UI would only cause more confusion so Mozmill just has a code editor (all tests in mozmill are in javascript, there is no other supported language).

I have to admit, I was pretty shocked by the kinds of feature requests we got after we implemented a recorder in mozmill. Windmill has a few orders of magnitude more users than mozmill and we got far more feature requests that bordered on *”magic”* in mozmill. Not mention that during the initial usage we got a complaint or bug report almost every time a recorded test didn’t play back and pass without any additional work. It wasn’t until today that I started to understand why we saw such a big difference in expectations.

One problem is that when you take a bunch of opaque background recorder magic and then just serialize it all to code it creates a context switch for the user. They didn’t watch the actions come in as they were being recorded so they have no idea where they came from and they stop and look at the code. Instead of a series of small steps, each interaction with the page creating a line of code, the user thinks of this as two steps; do stuff, test is written. This makes a lot of users disjointed from the editing process.

Another problem with generating code is that it’s expected to be valid. If a failure occurs the only thing to do is to show the exception information. The exception information highlights the line that has failed. Generated code is always hairy and difficult to understand so you just stare at it and are at loss as to how you should fix it. But the code isn’t where to look to fix this problem, the **page** is where you look to figure out what went wrong. The fact that you’ve gone from the exception information back to the code makes it less likely for you to move your attention **again** back to the page. This is where placement is a key factor as well and, if you can, you should also try to place the product side by side with the test interface.

With Windmill there is an entirely different workflow than mozmill. You record actions and they pop up in the IDE as you create them. Since the actions are created as you interact with the page it’s more transparent as to where they came from. Then you just play back the actions and as they run the editable tabs go red or green. This keeps the focus failure on the action without moving attention from code to exceptions and and allows you to watch the page as the passes and failures occur. Once a failure occurs it’s a little more likely now that you’ll see the cell go red before some ajax request finished rendering the piece of UI you tried to click on. Then, hopefully, you will intuitively add an action above the click that waits for the element to get rendered.

The mile high view of both recorders doesn’t make them seem much different. You *can* do the same set of tasks with each, but a series of small differences reduces the chances that you’ll fall off the intended pattern of use, which is to record a set of actions and use it as an outline to create fully working test. Those small differences also create entirely different expectations from your users.

The funny thing is, Adam and I have never thought about it this way. All we ever tried to do was keep things simple and keep the test creation, editing, running and debugging workflows integrated and this is just a happy side effect

Just in time for PyCon we’re releasing [Windmill 1.0](http://www.getwindmill.com/archives/412). It’s been almost 3 years of development and I’m both excited and relieved to have something we’re comfortable calling 1.0.

The latest RCs have seen bigger improvements than we thought would make it. A new wave of contributions has given us some great new features in Django support and test serialization and [Adam](http://www.adamchristian.com) has done some great UI work as well.

Big thanks to everyone who contributed to the release and I hope to see many of you at PyCon. Adam is doing a [talk on windmill](http://us.pycon.org/2009/conference/schedule/event/9/) which I’ll be helping out with and Adam and I will also be on a [functional testing tools panel](http://us.pycon.org/2009/conference/schedule/event/88/) which should be a lot of fun. And if you’re staying around for the sprint days of the conference we’ll be leading a [windmill sprint](http://us.pycon.org/2009/sprints/projects/windmill/).

I’ll be attending PyCon this year (finally!).

Somehow I haven’t attended the last 3 years that I tentatively planned to do so. 

Windmill will be well represented. Adam and I will be on a “Functional Testing Tools in Python” panel, and Adam will be giving a full blown Windmill talk.

There will also be a Windmill Sprint for anyone interested. If you want some hands on time figuring out how to test crazy webapp stuff we’ll be helping anyone who comes by the first few days of the Sprint. Our plan is to eventually transition the Sprint to working on the windmill2 codebase.

Feel free to stop by the Sprint or grab me during the conference if you want to discuss any of the other projects I work on (mozmill, jsbridge, pouch, etc) or if you want to discuss Mozilla testing and tools in general.

It’s usually a good idea to document cool features you write and tell people about them. I totally forgot about this one, which I wrote during DjangoCon a few months back.

A few years ago I wrote a patch for live server support in Django. The patch fell out of sync with trunk and was picked up a year or so later and substantially improved. But the patch didn’t end up making it in for 1.0 . Luckily the code is simple enough that I was able to use it windmill and add live server support to any Django 1.0 install by dynamically overriding and changing the classes that we need to start and stop a live test server for you django project.

After it’s all said and done this makes it dirt simple to bootstrap windmill tests from your Django unittests, all you need to do is define a single test class along side your other django unittests pointing to the directory of windmill tests you want to run and which browser you want to run them in and windmill’s django support will automatically start a live django server and run your windmill tests against it.

from os import path
from windmill.authoring import djangotest 
 
class WindmillTests(djangotest.WindmillDjangoUnitTest):
    test_dir = path.join(path.dirname(path.abspath(__file__)), 'windmilltests')
    browser = 'firefox'

This is also all written up on a wiki page along with some of the caveats.

I didn’t think we’d be doing any notable windmill releases until 1.0. Boy was I wrong!

Seriously Faster

On Wednesday Adam messaged me and said that the windmill startup time was too slow. He was right, we’ve know about this for a while but hadn’t put a lot of serious thought in to how we could reduce it.
The issue here was was that we have about 50 JavaScript files that need to get loaded for windmill to start.

Enter windmill-compressor, a new url namespace we added that concats all the js windmill needs in to one file and minifies it. We do this dynamically when windmill starts up, because adding a “build” step would just be too….. Java.

That reduced the startup time from 5-10 seconds to around 2 seconds. But that wasn’t good enough. We saw that most of that startup time was blocking waiting for the compressor to finish, so I threaded it when we start the windmill service and it does the compression while we wait for the browser to start up.

In all, windmill startup times are about 10x faster than 0.9!

Adam also decided to make our page load wait code a bit more aggressive which made not only startup times faster but all of our tests!

Native JavaScript Test Framework

We tied up the loose ends on the JavaScript testing framework and it can now shutdown all of windmill in it’s own teardown, hello continuous integration for native JavaScript tests

This was the last thing holding us back from promoting this along side the Python test authoring library.

We Love Firebug

In 0.9 we introduced Firebug Lite integration for windmill on all browsers. But when you’re using Firefox you probably still want to use the full Firebug extension, which is easy enough to integrate since we use mozrunner for Firefox launching.

Windmill now has a “firebug” command line argument that installs the full Firebug
extension when launching Firefox.