Wednesday, 18 June 2008

Testing dashboard

I like dashboards...both using them as well as developing them. If based on proper metrics they give a good picture of the current status in a single view. However, finding good metrics in testing is not a trivial task since counting test cases is not a good idea (especially for those of us who mainly do exploratory testing instead of executing scripted test cases).

I took the "Rapid Software Testing" course by James Bach about a year ago and he introduced me to "The Dashboard Concept" (see slide 138 in RST Slides). We had at the time a simular concept in an Excel sheet where colors were changed based on certain keywords but Excel has some limits when it comes to concurrent usage.

Now we are evaluting the use of my latest creation:

I have removed the Effort column from James' dashboard and I will probably add some extra columns for latest changed date and other things later on.

The dashboard aims at showing the test and quality status for an AUT, based on subjective conclusions for all test activities in a release/project.

If want to try it out, download it and place the files in C:\ if you don't want edit the html file. You also have to answer yes to all security warnings in order to get it to work.


Bas M. Dam said...

I like the database approach (my first idea was a tiresome Excel COM connection, but this is way mare elegant) and the one HTML file concept.

When you want to expand it, you could dive into mySQL and separate the HTML and the code by creating script files (this also improves performance).

I used to work with xml exports with real time (automated) test data, which I imported in a browser window with some AJAX, but that worked because we only needed disposable data.

Stefan said...

Thanks for your suggestions!

Actually my original is based on MS SQL Server instead of Excel and the functions are in a separate vbs-file. I converted it to an easy, portable "proof-of-concept" version for this post.

I store all my automation data in SQL Server (even steps nowadays!) for performance, easy access using SQL and for good traceability using primary keys. Though I consider exanding with "XML mode" allowing for "off-line" testing (typical a developer who run tests on a local machine).


Chethana said...

Hi Stefan,

I know i am posting question irrelevant to the topic discussed but i need ur help and valuable time
I was trying to execute Store Proc from QTP and find the method here and ur ans here

My Code is as follows:
Set dbConnection = CreateObject("ADODB.Connection")
dbConnection.ConnectionString = Environment.Value("ConnectionStr")
Set recordset = dbConnection.Execute("exec dbo.tp_pmt_inbound_process_ack '[99]','[2009-12-15T05:02:09]' ,'[GTUS]','[HrrC]','[my_ack_filename67.xml]','[2]','[pain.001.001.02]','[2009-12-15]','[1]','[ACCP]','[null]','[null]','[null]','[null]','[babkid]','[CT03554000000002]','[RA0000000002]','[ACCP]','[none]','[null]','[B40]','[null]','[1000]','[1500]','[GBP]','[2009-12-15T05:02:09]','[OMC]' ")

Do Until recordset.EOF
Msgbox recordset.Fields("[column]")
Set recordset = Nothing
Set dbConnection = Nothing

But its giving me an Error in the dbConnection.Execute method as follows:
"Error converting data type varchar to datetime."

Can u please help me regarding ...
my mailID

Stefan said...


I guess some parameter data in your Store Procedure call is invalid. Check the data type for each parameter. It is probably easier to do this in a query window in SQL Server Management studio.

Good luck!

Chethana said...

Thanks for the valuable input...