But regardless of where you put the class, if it’s in a test module that gets run it will be registered when the module is loaded. So if you put it in the folder your test is in, it should work, if not then something is wrong.

If something seems very off you can come on to IRC in #windmill on irc.freenode.net and someone should be around to help you debug it. Although right now during the holidays people seem to have been away a bit more.

I am not fluent at all in python so I don t know where I should put this __init__.py

I try putting it in the same folder than my test but it did not work.

The response is probably here “you add a simple class to the highest level __init__.py with your report code” but I don t understand the meaning of this sentence.

If the localization testers want a different API then we can write one specifically for that use case, but having the default API for functional test developers be by position or id requires them to know more about the menu definition internals than any of our functional testers know.

Not having an abstraction API like this one for your specific needs doesn’t block you from testing the menu’s the way you would like, you’ll just need to pull out the elements on your own. You can look at the menu API code to get you on your feet;

http://code.google.com/p/mozmill/source/browse/trunk/mozmill/extension/resource/modules/controller.js#115
http://code.google.com/p/mozmill/source/browse/trunk/mozmill/extension/resource/modules/controller.js#141

I have a testing extension that opens all menu items one by one, and being able to do that in a cross platform manner would be cool. But I’d need to have it in a way that’s independent of the localization used.

Same would go for testing localized extensions, something I’m thinking about as well.

That said, simulating user interaction well is in fact something that we’re missing from our testing story (it’s possible to do a bit of it in mochitest, but it’s cumbersome, often buggy, and certainly painful to use for things like popups or menus), and it’s great to hear that this will fill that gap! This certainly answers my question about what the goal is.

As far as covering the gaps in mochitest and our other tests… those gaps aren’t due to boundaries per se, certainly not in mochitest. All sorts of things _can_ be accomplished in mochitest right now (e.g. one could create signed jars on the fly). The real problem is the lack of helpers to make those tasks easy to accomplish, which prevents effective writing of tests that exercise certain parts of the codebase.

MozMill tests don’t need to be defined “as chrome” (in the chrome manifest) in order to run in the privileged environment, the scripts are imported and run in that environment dynamically.

MozMill was originally designed for funcitonal testing, it has a very large API set for simulating user interactions. Taking on unittesting is new to MozMill, this is our first set of features specifically targeted at unittesting. You can also mix and match unittesting and fucntional testing in MozMill (simulate some user interactions to trigger a certain code path, then use the jum to assert various properties).

The main advantage to using MozMill is that anyone in the community can install the extension and write/run/debug test scripts with ease against any release, debug, or nightly build. When tests are run in continuous integration the Python side (using jsbridge) integrates seamlessly but Python is not required to write/run/debug tests.

I’m not familiar enough with the problems you’ve described with mochitests to say whether MozMill handles them or not. MozMill’s environment is intended to be simple and flexible in order to avoid most of the problems that come with creating big “framework” environments for testing, so it’s possible that the boundaries you’re seeing with mochitest don’t exist in MozMill but I can’t tell without more knowledge about the depth of the problems.