Traceback (most recent call last):

I’m in London for the next few days and would love to grab a drink with any community members be you Mozilla, CouchDB, Python, Windmill, JavaScript or just plain old coffee, whisky or beer geeks

I’m getting all packed up and leaving Sunday for EuroPython in Birmingham, UK.

This will be my first time at EuroPython and my first time in Europe!

I’ll be giving two talks, one on Windmill and one about CouchDB and Python. The Windmill talk will be more or less the talk that I gave at Open Source Bridge last week, which went very well. This is the first time I’ll be talking about CouchDB, the most exciting new technology on the web. The talk will mostly be about breaking our old data modeling habits that we developed to deal with SQL and what libraries and tools are available for interacting with CouchDB in Python.

I will also be in London for a few extra days after the conference so anyone interested in a meetup should ping me.

I’ll be leaving tomorrow morning for Open Source Bridge in Portland, Oregon.

I’m putting together a new Windmill talk that tries to incorporate all the feedback we’ve received over the last year of speaking which I’ll be presenting on Thursday.

Mozilla is also a sponsoring the conference and there is going to be some great Firefox related sprints in the hacker lounge. Dietrich is also giving what sounds like an awesome talk on extending Firefox called Firefox Switchblade.

Hope to see you all there!

PS. I’ll also be at EuroPython and the Community Leadership Summit, more on those later

I had some fun this weekend with Python, <video>, CouchDB and Brett Gaylor’s RiP: A Remix Manifesto. In just a few hours I was able to crank out a little annotations remix which allows anyone to add annotations to the film that are displayed as people view it.

I’m hosting it on my little mac mini (currently hidden in a data-center) so hopefully it doesn’t fall over pushing so much video

I’ve posted all the code up on github. The more I use <video> and CouchDB the more excited I get about the future of web applications. This entire project was done in little chunks of spare time over the weekend and most of that was me messing around with styling. To get the data stored, queried, and displayed took less than 2 hours.

Hope you all enjoy the annotations remix and if you haven’t already go and pay what you want for a terrific copy of RiP: A Remix Manifesto. It’s worth it.

The Rails community seems to be on the verge of imploding, not necessarily because of a particular event but because of the response by Rails leadership to the event and their lack of meaningful reflection on why this happened.

This event has been well commented on, so I won’t go in to all the details, but unsurprisingly the best comments I’ve read so far have come from women, in particular Audrey Eschright. It’s good to see meaningful discussion in that community but I think the proposed solution of increased “professionalism” is counterproductive and generating an even more hostile response from Rails leadership.

Professionalism doesn’t have a good history of promoting equality. After segregation “professional” was one of many code words for “white male” and even after many years when non-white males began entering the work force the idea of professionalism served to strip people of their culture and identity to better fit in to a white male dominated environment. For those of us in open source, and especially those who have left traditional corporations, we identify “professionalism” as the way corporations turn people in to plastic dolls in suites that can talk to other plastic dolls from other corporations at trade shows without getting “off message”. Luckily, we aren’t talking about a corporation here, we’re talking about a community and community isn’t about professionals it’s about people.

I think people see professionalism as a solution to this problem because it provides a way for us to change people’s conduct that we disagree with without actually changing people or how they think they should conduct themselves in a community. The real problem here is that some people think it’s just fine to engage in public conduct that knowingly excludes people and doesn’t help include or encourage any new participation.

When news of this little outburst broke I have to say I wasn’t surprised that it happened in the Rails community. All tech communities have a low number of female participants, this is something nobody is great at yet, but for some time now Rails has been the worst offender. In the early days Rails was promoted by DHH in what, at the time, sounded like a great strategy: Attack dominant technologies (Java) and piss people off that use them long enough to check out your stuff. This strategy generated huge buzz and lead to one of the fastest adoptions of a new technology I’ve ever seen and it’s worth noting that the technology was actually so good that people could go from being a pissed off Java developer to a Rails developer in a relatively short amount of time.

But the biggest problem with DHH’s strategy was that it was hostile and it intentionally excluded a large number of people who were invested in the old technology and couldn’t get over being pissed off at DHH. The first thing DHH will say is “well, we don’t want or care about those people”, and he’s right, Rails doesn’t need those people. But when you, as the leader of a project, are hostile and partake in action that intentionally excludes people, regardless of who those people are, it creates an environment where others see no need to alter their behavior not to exclude people.

I’m really disheartened by DHH’s “I’m an R rated individual” post, because it’s not that he’s wrong, he’s actually right, he should be able to be an individual in the community he’s built, but if he doesn’t intentionally curb some of his behavior he’s going to encourage exaggerations of the worst parts of his personality by the example he sets. It’s great that DHH loves Louis C.K and his dick jokes, but that doesn’t mean he would intentionally start a keynote with them, he knows better, and that’s what makes the difference and that’s what is so wrong with Matt’s talk, he knew before he went up there that this would make a lot of people uncomfortable and discourage participation from women and he did it anyway.

In order to lead an inclusive community you have to encourage participation and allow everyone to be individuals. If you treat everyone with respect, no matter who they are and no matter what they do, others will conduct themselves the same way. It’s always easier to respond to hostility with hostility but all you’re doing is creating an environment where hostility is the norm and it’s alright to exclude people.

The kind of communities we want are not impossible, in fact we get closer and closer every day. I’m lucky enough to be at #moz09 this week, and I’m looking around at over 200 people from different cultures from around the world none of which seem to be trying to hide who they are or where they came from for some notion of “professionalism” and everyone treats each other with respect and every day we gather more participation from a broader set of individuals. We aren’t perfect, we’re never where we want to be, but we should be confident we’re going in the right direction.

I’ve been writing some automated tests for the new <video> work in Firefox 3.5 and it’s been incredibly fun. I spent some time in the trenches at Real Networks and dealt with accessing embedded video in the browser so I really appreciate how much nicer <video> is than embedded players like Real and Flash.

People talk a lot about embedded plugins being “stuck in a box” and while this is partially true I don’t think it’s descriptive enough. Embedded players like RealPlayer and Flash open up methods that can be called from javascript, so you do have some ways of interacting with the box, but it’s limited to somewhat rigid pre-defined interfaces. It’s also worth noting that using these interfaces just doesn’t feel like web development the way doing regular JavaScript and DOM feels. It’s not dynamic and you’re limited to only doing what the plugin authors have already thought of. The beautiful thing about the browser as a platform is that it enables usage far beyond what the original creators have intended.

Using <video> feels like the web. You resize it by setting size attributes the same way you would an interface node. You seek by setting the currentTime attribute, you can listen for the time changes to create your own interface by adding a listener for “timeupdate” events. You can write interfaces around <video> without switching contexts because it’s not a foreign technology jammed in to the browser. Here is some javascript that implements a few video controls.

var v = document.getElementById("v1");
document.getElementById("playDiv").onclick = function(e) {v.play()}
document.getElementById("pauseDiv").onclick = function(e) {v.pause()}
document.getElementById("reloadDiv").onclick = function(e) {v.load()}
document.getElementById("seekDiv").onclick = function(e) {
    v.currentTime = document.getElementById('seekField').value;
    }

Isn’t that beautiful! Maybe I’m overly excited because I’ve had to deal with this using older technologies but this just feels good, and if it feels good then I know people are going to get excited about it and build cool stuff.

I’ve started stewing over Garrett Murray’s post on negative user reviews for his Ego application for iPhone that recently showed some breakage when Google Analytics changed up. What started to really upset me was one of his closing paragraphs.

This kind of thing continually reinforces something I’ve thought about a lot since the App store was released, which sounds horrible to say but it might be true: Apple is creating an ecosystem of the kind of customers I don’t want. With the ridiculous approval process leaving bugfixes to take over a week to show up, with prices being driven down to nothing by farting apps… it just feels hostile to me. While I have plenty of great customers who have been raving about the app, all it takes is one little issue and it all comes crashing down.

It’s no longer acceptable for consumer applications to “blame the user”. Sure, many people still do it, but if a blog post like this just blamed individual users for their complaints without this closing paragraph it wouldn’t be gaining so much steam and I probably never would have read about it. Instead Garrett blames the community of users, and by extension Apple for process that encourages this kind of community.

The first rule of community leadership in open source is simple; “Don’t be an asshole.”. While simple, sometimes it’s the hardest rule to follow. Many people in the community can be difficult, some of them downright hostile, but you can’t be hostile back and you can’t be an asshole. When you’re hostile or an asshole in your community you give license to the community to do the same. You cultivate the behavior that antagonized you in the first place. This is the rule when trying to lead contributors and I have to assume it’s the rule when trying to lead users and posts like this violate the rule by blaming users for their own expectations and then dismissing them altogether.

He tries to deflect attention to Apple since they are the gate keeper. Dismissing the bulk of your user community due to the gate keeper is logic that is so convenient you know it has to be flawed. The truth is most platforms have some kind of gate keeper, this is the world that we live in, but your complaints about your users have nothing to do with Apple.

iFart is 1 dollar but you’re giving it way too much credit if you think that’s why iPhone users have lower price expectations. You know what else is 1 dollar, Ocarina. Applications and games like Ocarina are larger triumphs of creativity than they are of engineering time. Not to say they are easy, or simple, but they are priced cheaply because the creators are confident in their product and because they didn’t spend a year developing it. This is where iPhone users have cultivated their expectations about interface, graceful failure, and low prices, because of your competitors not because of the gate keeper.

But even if all the user community’s expectations and demeanor is Apple’s fault, even if your users are cheap and vicious and given way too big a soap box, it’s still your fault.

As an example. Firefox get’s requests for features from Windows users to make certain things be “more like Windows”. If you look at the Firefox interface over 3 operating systems you’ll notice that the preferences menu item moves around based on which platform it’s on. The truth is, if you designed an interface from the ground up in some ideal world there would be one place that is best to put the preferences item, but that’s not the world we live in, we live in a world of expectations from user communities. The easiest thing to do is to say “we don’t need those users” or “those users are stupid and don’t understand proper design” but it’s not the right thing to do because it’s not what they want.

What your users are really saying with those complaints is that this failure frustrated them to the point of opening up iTunes and writing a complaint, not just that something broke. You could have failed more gracefully, you could have presented them with a button that opened a support web page with more detailed information and assurances that it was being resolved. Then after some time the bad reviews would get swallowed up by the positive ones you mentioned. But now you don’t have to do anything because you’ve convinced yourself that you did nothing wrong and it’s just a bunch of users you don’t want complaining about something they are too dumb to understand.

I hope some day Garrett finds his perfect user community with expectations that match his own. I’ll stick to working on products for everyone else, listen to their feedback seriously and try to improve them.

So much good stuff landed in Windmill over the last few weeks that we decided to push another major release.

The biggest new features are:

• django management command for running windmill tests (Jacob said the existing django support wasn’t good enough and I agreed so I wrote this during the PyCon Sprints)
• new nose plugin
• cygwin support contributed by Simon Law (he went and wrote an implemenation of winreg for cygwin to get this to work).

There were also some really good bug fixes that landed:

• much better unicode handling and serialization (adam)
• fix for POST to foreign domains (Anthony Lenton)
• continued improvements to click simulation (adam)

The release is up on PyPI and you can install/update with:

$ easy_install -U windmill

For anyone interesting in working on windmill we’re having a Sprint in #windmill on irc.freenode.net tomorrow April 8th, 2008 for pretty much all day. We’re going to be improving the unittests for Windmill itself.

The next planned major release will be 1.2 which will include the much anticipated SSL support, courtesy of some great work being done by Anthony.

The video from PyCon of the Functional Test Tools Panel has been making the rounds and one comment in particular that seems to be gaining some traction is a comment Jason Huggins made about test recorders being “evil”. I didn’t comment about it while I was on the panel because I have mixed experiences with recorders in the tools I work on and some of those experiences have left me feeling the same way.

I agree with Jason that recorders that generate code with the expectation that you will be able to just run that code as a test are definitely evil. Aside from just not working most of the time they create false expectations about test tools. But, I disagree that they can’t be implemented in a way that is useful and side steps a lot of those false expectations. Note: On twitter Jason has added that he agrees with my disagreement probably for the same reasons I’ll highlight below.

Looking at two tools I’m still an active developer on, Windmill and mozmill, both have a recorder but one I find highly successful and the other I’ve struggled greatly with.

The recorder in the Windmill IDE is phenomenal and almost entirely due to the work that Adam has put in to it. The entire test authoring UI in Windmill is action based meaning there is a series of interface simulations you can do with a single variable (an element lookup or in some cases a string to eval). Each action is a cell that can be moved around and new actions added above or below. The same UI is leveraged for running and debugging tests, each cell shows green or red if a test has passed or failed.

The reason I like this UI and similarly simple UI’s like FireUnit’s UI is that they leverage the relative simplicity of the test APIs. There is no code in the Windmill IDE, just actions, so when you record a set of actions they don’t show you code just the actions as you interact with the page.

Now look at the mozmill UI. Mozmill suffers from some additional necessary complexity. In mozmill we have to test multiple windows, which requires a more complex API and some required setup steps. Mozmill can’t have the same action” based UI because we have two additional variables to every UI interaction, the window (which usually requires an additional API to pull out) and the document (also requiring an additional API for easy access). Removing the complexity of the API from the UI would only cause more confusion so Mozmill just has a code editor (all tests in mozmill are in javascript, there is no other supported language).

I have to admit, I was pretty shocked by the kinds of feature requests we got after we implemented a recorder in mozmill. Windmill has a few orders of magnitude more users than mozmill and we got far more feature requests that bordered on “magic” in mozmill. Not mention that during the initial usage we got a complaint or bug report almost every time a recorded test didn’t play back and pass without any additional work. It wasn’t until today that I started to understand why we saw such a big difference in expectations.

One problem is that when you take a bunch of opaque background recorder magic and then just serialize it all to code it creates a context switch for the user. They didn’t watch the actions come in as they were being recorded so they have no idea where they came from and they stop and look at the code. Instead of a series of small steps, each interaction with the page creating a line of code, the user thinks of this as two steps; do stuff, test is written. This makes a lot of users disjointed from the editing process.

Another problem with generating code is that it’s expected to be valid. If a failure occurs the only thing to do is to show the exception information. The exception information highlights the line that has failed. Generated code is always hairy and difficult to understand so you just stare at it and are at loss as to how you should fix it. But the code isn’t where to look to fix this problem, the page is where you look to figure out what went wrong. The fact that you’ve gone from the exception information back to the code makes it less likely for you to move your attention again back to the page. This is where placement is a key factor as well and, if you can, you should also try to place the product side by side with the test interface.

With Windmill there is an entirely different workflow than mozmill. You record actions and they pop up in the IDE as you create them. Since the actions are created as you interact with the page it’s more transparent as to where they came from. Then you just play back the actions and as they run the editable tabs go red or green. This keeps the focus failure on the action without moving attention from code to exceptions and and allows you to watch the page as the passes and failures occur. Once a failure occurs it’s a little more likely now that you’ll see the cell go red before some ajax request finished rendering the piece of UI you tried to click on. Then, hopefully, you will intuitively add an action above the click that waits for the element to get rendered.

The mile high view of both recorders doesn’t make them seem much different. You can do the same set of tasks with each, but a series of small differences reduces the chances that you’ll fall off the intended pattern of use, which is to record a set of actions and use it as an outline to create fully working test. Those small differences also create entirely different expectations from your users.

The funny thing is, Adam and I have never thought about it this way. All we ever tried to do was keep things simple and keep the test creation, editing, running and debugging workflows integrated and this is just a happy side effect

Just in time for PyCon we’re releasing Windmill 1.0. It’s been almost 3 years of development and I’m both excited and relieved to have something we’re comfortable calling 1.0.

The latest RCs have seen bigger improvements than we thought would make it. A new wave of contributions has given us some great new features in Django support and test serialization and Adam has done some great UI work as well.

Big thanks to everyone who contributed to the release and I hope to see many of you at PyCon. Adam is doing a talk on windmill which I’ll be helping out with and Adam and I will also be on a functional testing tools panel which should be a lot of fun. And if you’re staying around for the sprint days of the conference we’ll be leading a windmill sprint.