Friday 13 March 2009

Test framework - Stop feature

I implemented a new feature in my automation framework today...

My test suites are usually very long scenarios and sometimes it takes hours to get to a test case which is by the end of a scenario. Imagine creating new test cases at the end, running the suite and a script failure occurs which in worst case require total rerun (that is also a major reason why I have a 'Resume' test feature)...

...so I thought a stop feature would be nice to have so I implemented a new state/status for a test case run status; 'Stop'

It work like this: Once a test run is created (more clues about my data model will be published here within a month or so, I hope) you set test case run status to Stop for a test case (via Report GUI) and the test execution will stop when the test case is up next in the test case execution queue (and sending an e-mail about it). Now, you can resume the test in debug mode and fix errors on the fly (remember to change status to 'No run' first). Or perhaps backup the AUT database before you continue?

That's it, not that complicated (I wonder why I haven't thought of it before)...

6 comments:

Giorgio said...

It is not possible to run the test in question in isolation?

Stefan Thelenius said...

Mostly yes, fortunatly...

However my automation design is based on creating new test data for each test suite run session and does not require a certain test data start set. So even if verification of the test case in isolation is possible, additional verification during the first test run also is necessary in order to verify robustness etc. And in worst case a script failure results in a necessary rerun from the start...therefore a database snapshot prior test case execution or possibility to run the test case in debug mode could save you some time (of course you could use breakpoints as well).

MacroTesting said...

Good article, i really like it. I am doing a bit on research about software testing and i found also macrotesting www.macrotesting.com to be very good source.

Thanks for your article

Regards,
Prem

Stefan Thelenius said...

Thanks

Anshoo Arora said...

This is an excellent article, Stefan! It has happened on this project on numerous occasions that I was asked to execute my test suite during maintenance or a down-time. So, as you would expect, everything fails and the mailbox is full of hundreds of mails detailing the same error message, over and over again. This will really help in reducing such events from occuring again. Thank you!

Stefan Thelenius said...

Thank you :-)