QTP: QuickTest Professional Unplugged by Tarun Lalwani

Tarun Lalwani's book QuickTest Professional Unplugged covers QTP basic as well as advanced features.

I have used QTP since 2001 (version 6.x) and if I have had this book from the start, a lot of hours and frustration had been saved.

Tarun has prior the book, contributed to the QTP community with numerous of high quality articles and forum post replies over the years. I recall an old article about Descriptive programming (DP) which made me switch from OR to DP Big-bang style (I don't regret it...).

My favorite parts in the book are the chapters regarding RegExp, IE/Word/Excel objects, HTML DOM and of course the advanced section at the end. But this book include a lot of other QTP/VBScript features as well and is well suited both for newbies (you should have some programming experience at least) and advanced users.

This is a must have for any person using QTP and/or VBScript-based automation!

Watij: Brief progress report

I have been evaluating Watij lately and here are my conclusions so far for some automation features important to me:

IDE

Since I am from the QTP/VBScript world I still struggle a bit with the Java syntax and OO...furtunatly my IDE (IntelliJ IDEA) has very good refactoring features (and other develop-friendly features as well) so it is not necessary to make perfect code/structure from the start.

GUI test execution speed

Not as fast as QTP but very close. Since the source code is available I managed to change a hard-coded timeout property and I guess there are several others that you could tweek for faster performance.

Test object model

Quite simular to Descriptive programing in QTP.

QTP example: Browser("Google").Page("Google").WebEdit("q").Set "Watij"
Watij example: ie.textField(name,"q").set("Watij");

Have not explored all possibilities with dynamic test object handling in Watij yet but at least getting all links in a frame into an array was very simple.

Community

The QTP community is awesome and Watij's dito seems ok but rather small.

Pop-up handling

QTP is very good handling pop-ups but have a weakness running such actions in a locked workstation. Watij has so far also been very good at this (except for an authentication dialog which I handled with a quick and dirty java robot solution) and IT WORKS IN LOCKED MODE! This feature alone allows unattended GUI tests being run 24x7...This seems to good to be true so I will do some more testing on this using remote desktop -> server...

Test framework - Multiple tool support

To design a robust and cost effective automation framework is a complex task...

This morning a read a good post about Programming Paradigms in Test Automation which I can recommend reading.

I have on my agenda to switch from VBScript to Java in my framework. I have realized that this process is going to take a while since I have not yet found a decent tool for the GUI testing part...

I am currently evaluating Watij and so far it seems really promising since it can deal with pop-ups much better than Selenium. However, test execution speed is not that good when running Watij (QTP or worse) from IDEA so next step is to see how it handles stand-alone execution...stay tuned

Since I want to support multiple tools/langauges in my framework I have decided to move all SQL from code to Store procedures for easier maintenance. Here is an overview picture of how it will look like when implemented:

VBScript: Sending e-mail

Sending e-mail automaticly at various test contexts could be a useful feature (beware of "spam" if you are sending mail to a shared address...) so as a follow-up to this post: http://abouttesting.blogspot.com/2008/01/cdomessage.html I thought I would share an example on how to send an HTML-based e-mail using CDO:

'Sending E-mail sample

strEmailSubject = "Testing CDO"
strMessageFrom = "myemailfrom@mydomain.com"
strMessageTo = "myemailto@mydomain.com"
strHTMLBody = "<B>Bold</B>&nbsp;example"
strSMTPServer = "mail.mydomain.com"

SendEmail strEmailSubject, strMessageFrom, strMessageTo, strHTMLBody, strSMTPServer

Function SendEmail (strEmailSubject, strMessageFrom, strMessageTo, strHTMLBody, strSMTPServer)
Set objMessage = CreateObject("CDO.Message") 
objMessage.Subject = strEmailSubject
objMessage.From = strMessageFrom
objMessage.To = strMessageTo
'The line below shows how to send using HTML included directly in your script
objMessage.HTMLBody = strHTMLBody
Const cdoSendUsingPickup = 1 'Send message using the local SMTP service pickup directory. 
Const cdoSendUsingPort = 2 'Send the message using the network (SMTP over the network). 
Const cdoAnonymous = 0 'Do not authenticate
Const cdoBasic = 1 'basic (clear-text) authentication
Const cdoNTLM = 2 'NTLM
'==This section provides the configuration information for the remote SMTP server.
objMessage.Configuration.Fields.Item _
("http://schemas.microsoft.com/cdo/configuration/sendusing") = 2 
'Name or IP of Remote SMTP Server
objMessage.Configuration.Fields.Item _
("http://schemas.microsoft.com/cdo/configuration/smtpserver") = strSMTPServer
'Type of authentication, NONE, Basic (Base64 encoded), NTLM
objMessage.Configuration.Fields.Item _
("http://schemas.microsoft.com/cdo/configuration/smtpauthenticate") = cdoAnonymous
'Server port (typically 25)
objMessage.Configuration.Fields.Item _
("http://schemas.microsoft.com/cdo/configuration/smtpserverport") = 25 
'Use SSL for the connection (False or True)
objMessage.Configuration.Fields.Item _
("http://schemas.microsoft.com/cdo/configuration/smtpusessl") = False
'Connection Timeout in seconds (the maximum time CDO will try to establish a connection to the SMTP server)
objMessage.Configuration.Fields.Item _
("http://schemas.microsoft.com/cdo/configuration/smtpconnectiontimeout") = 60
objMessage.Configuration.Fields.Update
'==End remote SMTP server configuration section==
objMessage.Send
End Function

You can download the script here:
http://abouttesting.fileave.com/SendEmail.zip

Test framework - Multi-threading

From http://abouttesting.blogspot.com/2008/06/test-framework-brief-description.html

Known limits:
Parallel execution is not implemented (you cannot use multiple test tool instances for a test run in order to decrease test execution time).

Check. Implemented today :-)

Solution: Call a Store procedure with test run id and test run option id (including test tool type) and it returns next compatible test case in the execution queue including Go/NoGo (if NoGo wait until Go critera is met).

Is this a necessary feature?

I have many test suites that are End-to-end based and spans over several test days. Since each day requires a lot of batch processing (included in test suite as well) you want to combine a lot of test scenarios in one test suite instead of having one test suite/test scenario. The largest test suite has now over 900 test cases (multi steps)...so anything that reduces overall execution time is very welcome :-)

Test data model part 2

A snapshot document of my test framework data model is now available for download:

http://abouttesting.fileave.com/TEST_DATA_MODEL_090420.zip

What to use it for?

I think it mainly can serve as an inspiration source on how to (or how not to...) model your test framework as it is difficult to use it "as is" since it requires SQL logic (in code or in store procedures) and a bunch of GUIs (some has been published earlier).

When I read the document I realize that some things should be refactored (bit instead of varchar for True/False columns for example) but all design decisions made sense to me at the time of creation. The database is "hand-made" from scratch using SQL Server GUI to create and change tables. Columns and tables have been added from time to time ever since.

One of the reason why I choosed to use SQL Server instead of excel was that my AUT required of lot of parameters...if I do select distinct test_case_parameter_name from test_case_parameter the answer is 192...

Test framework - 'stop if too many errors feature'

Another feature I implemented a few weeks ago is 'stop if too many errors'.

Since an e-mail is sent for each test case failure, the e-mail inbox could be flooded if for example, an unhandled test environment issue occurs or the AUT build is completely broken. So to prevent that I have a parameter for how many test case failures that are tolerated before stopping a test run. Not that complicated either but good to have (especially if the e-mails are sent to a shared address...)

:-)