Adam wrote a great post recently on some code he wrote that outputs his test results in an xml format that Hudson can digest. It serves as an example of a fairly underutilized feature in functest for reporting. It’s incredibly simple and powerful, you add a simple class to the highest level __init__.py with your report code and it will be called when all the tests have finished.
Ideally you want to checkin your reporting mechanism with the source for all your tests but only run the reporter when it runs in continuous integration. Another functest feature is that any unrecognized command line = arguments are stuffed in to dictionary at functest.registry, and there is code in the windmill command line that will do the same.
$ windmill shell report=true [14:15]
report is not a windmill argument. Sticking in functest registry.
In [1]: import functest
In [2]: functest.registry
Out[2]: {'report': 'true'}You can use this in your Report class to decide when you should actually report, you can even do different types of reporting based on what you pass. I wrote an example by refactoring Adam’s earlier Hudson example, I’ve also sped up some of the code and use ElementTree to write the xml instead of writing the string output by hand.
import functest from functest import reports from xml.etree import ElementTree class JUnitReporter(reports.FunctestReportInterface): def summary(self, test_list, totals_dict, stdout_capture): if functest.registry.get('report', False): total_sec = reduce(lambda x, y: (y.endtime - y.starttime).seconds + x, test_list, 0) e = ElementTree.Element('testsuite') e.attrib['errors'] = e.attrib['failures'] = str(totals_dict['fail']) e.attrib['tests'] = str(len(test_list)) e.attrib['name'] = 'windmill.functional' e.attrib['time'] = str(total_sec) for entry in test_list: t = entry.endtime - entry.starttime test = ElementTree.Element('testcase') test.attrib['classname'] = test.attrib['name'] = entry.__name__ test.attrib['time'] = str(t.seconds)+'.'+str(t.microseconds) if entry.result is not True: failure = ElementTree.Element('failure') failure.attrib['type'] = entry.tb[-1].split(':')[0] failure.text = '\n'.join(entry.tb) test.append(failure) e.append(test) if len(stdout_capture): # ElementTree doesn't support CDATA so we need to hack it a little replace = '#$!#$!replace#@!$' sysout = ElementTree.Element('system-out') sysout.text = replace e.append(sysout) outs = ElementTree.tostring(e).replace(replace, '<!--[CDATA['+stdout_capture+']]-->') else: outs = ElementTree.tostring(e) f = open('continuous_test.log','w') f.write(outs) ; f.close() reports.register_reporter(JUnitReporter())
[...] UPDATE: Mikeal has posted some additions/revisions to this code sample, here. [...]
[...] from functest import reports from datetime import datetime class JUnitReport(reports.FunctestReportInterface): def summary(self, test_list, totals_dict, stdout_capture): total_sec = 0 for entry in test_list: time_delta = entry.endtime - entry.starttime total_sec += time_delta.seconds out = ‘n’ out += ‘n’ for entry in test_list: if entry.result is not True: entry_time = entry.endtime - entry.starttime out += ‘n’ out += ‘n’ #out += str(stdout_capture) #until I can figure out how to get the traceback out += ‘There was an error in ‘+ entry.__name__ out += ‘nn’ out += ” else: entry_time = entry.endtime - entry.starttime out += ‘n’ out += ‘nn’ out += ” f=open(’continuous_test.log’,'w’) f.write(out); f.close() reports.register_reporter(JUnitReport()) Happy automating! UPDATE: Mikeal has posted some additions/revisions to this code sample, here. [...]
[...] You can also read a great entry about adding reporting to your tests on Mikeal Rogers blog, here. [...]
[...] You can also read a great entry about adding reporting to your tests on Mikeal Rogers blog, here. [...]