"...test data independant"

"...test data independant" (from post last week)

What do I mean by that ?

In automation, I try to avoid hard-coded test data which depends on whether the AUT database is in "start-position" or not.

Mostly I create test data during run-time using random values or getting random data from the AUT database which match my critera for the specific test case.

Consider a test case where you want to change an address for example. Here is an approach you could use (in pseudo-code]:

Select top 1 [myAddressColumns] from ADDRESS where [myCritera = True] order by newid() 'Note! Newid() MS SQL Server specificOptional
If blnNoAddressFound Then call createAddress 'In case of empty database call test case/method for create an address

Advantages:
Test case can be executed to any AUT database (empty or full)
You get test data variation using random values

Disadvantages:
It is more difficult to implement compare methods (but not impossible)

SQL - a tester's best friend?

If you work with applications which has a database, good skills in SQL are required in order to be an efficiant tester (unless you sitting next to a friendly DBA).

With SQL (and other database operations) you can:

1. Find, change, create or delete test data
2. Do backup and restore of databases
3. Plus a lot of other useful things, for example you learn much about the application through the database/data model

If you cannot perform 1 and 2 by yourself you will encounter serious bottlenecks in your testing progress.

So, what you need are:

1. Learn basic SQL (select)
2. Get proper account settings in your test environment(s) database(s) - read only
3. Learn advanced SQL (join, update, insert, store procedures, backup/restore, temp tables etc)
4. Get proper account settings in your test environment(s) database(s) - SA or at least anything close

Beware, you must prove yourself worthy for the DBA to achieve no. 4

I went from stage 1 to 4 in a few months just by self-study articles and other people's SQL. I am not considering myself being a skilled DBA but for my testing purposes I am doing quite OK. Also in automation knowing SQL is priceless in order to get your automation test case dynamic and testdata independant. Further more you can build a lot of AUT specfic test tools which will improve your testing dramaticly.

Over and out...

ps.

Here is a list of my favourite database engines which I have worked with in my testing over the years (ranked from a user friendly perspective):

1. Microsoft SQL Server 2005
2. Oracle (don't remember version no)
3. DB2 (don't remember version no)

VBScript: Reading text files

Until now I have used the FileSystemObject (FSO) when reading text files in automation but this week I wanted to read a utf-8 encoded file which did not work using FSO. I used my standard approach dealing with this kind of problem:

1. Collect information about the issue (mainly using Google)
2. Analyze information and design solution
3. Implement solution

So I found another COM-object which can be used reading files which seems to manage most of the Char Sets I normally deal with: Adodb.Stream

The result (getFileContent.vbs):

Function getTextFileContent (strFileName, strCharSet)
Const adTypeBinary = 1 'not used
Const adTypeText = 2
'Set default CharSet
If strCharSet = "" Then strCharSet = "ASCII"
' *** CharSets ***
'    Windows-1252
'    Windows-1257
'    UTF-16
'    UTF-8
'    UTF-7
'    ASCII
'    X-ANSI
'   iso-8859-2
Set objStreamFile = CreateObject("Adodb.Stream")
With objStreamFile
    .CharSet = strCharSet
    .Type= adTypeText
    .Open
    .LoadFromFile strFileName
    getTextFileContent = .readText
    .Close
End With
Set objStreamFile = Nothing
End Function

QTP: Debug class for Dictionary

As I mentioned earlier, I use the Dictionary object a lot in my test framework. However it requires some extra efforts when debugging in QTP compared to ordinary variables since your Keys and Items are not visable by default in Debug viewer's Variables tab. You either manually add Keys to the Watch tab or add extra code where your Dictionary Keys and Items are stored in an array:

arrKeys = objDictionary.Keys
arrValues = objDictionary.Items

The last couple of days I have tried to learn a bit about Classes in VBScript and I noticed that a class public variables are available in QTP's Debug Variables tab in alphabetic order. That made me come up with this crazy idea to maybe replace the use of Dictionary object with a custom class instead...but Classes in VBScript seems a bit limited according to my requirements. However, for debugging purposes it could be nice to create a "Class copy object" of a Dictionary:

'Create a dictionary object and add some parameters
Set objDictionary = CreateObject("Scripting.Dictionary")
objDictionary.Add "A_Parameter","47"
objDictionary.Add "Z_Parameter",""
objDictionary.Add "C_Parameter","11"
'Create a class out of the dictionary and a new debug object
Execute CreateClass("DebugDictionary", objDictionary)
Set objDictionaryDebug = New DebugDictionary
'Display A_Parameter value
Msgbox objDictionaryDebug.C_Parameter
Set objDictionaryDebug = Nothing
Set objDictionary = Nothing


Function CreateClass (strClassName, objDictionary)
'* Create Class string from Dictionary object
'* By Stefan Thelenius 2008-04-02
'* Usage example: Execute CreateClass("MyClass", objDictionary)
'Get keys and values into arrays
arrKeys = objDictionary.Keys
arrValues = objDictionary.Items
For i = 0 to UBound(arrKeys)
    'Create Public variables
    strClassPublicVar = strClassPublicVar & "Public " & arrKeys(i) & vbLf
    'Set default values to variables
    strClassPublicInit = strClassPublicInit & arrKeys(i) & "=""" & arrValues(i) & """" & vbLf
Next
'Add number of Keys as Count Int
strClassPublicVar = strClassPublicVar & "Public Count" & vbLf
strClassPublicInit = strClassPublicInit & "Count=" & UBound(arrKeys)+1 & vbLf
'Set Class and initializer
strClassStart = "Class " & strClassName & vbLf
strClassEnd = "End Class"
strClassInitStart = "Private Sub Class_Initialize" & vbLf
strClassInitEnd = "End Sub" & vbLf
'Return class string
CreateClass = strClassStart & strClassPublicVar & strClassInitStart & strClassPublicInit & _
    strClassInitEnd & strClassEnd
End Function

Above code example is available for download here: http://abouttesting.fileave.com/DictionaryClass.vbs

When using QTP the debug view looks like this:

XML automation - Part V - Validate XML response

This is the last part...but I will try to summarize the posts in an article later on...maybe I will add more features, for example how to handle attributes and multiple elements.

All the example code and files are available in compressed format here for download.

Please note that I have changed the test case parameter ConvertTempResult to #ConvertTempResult in order to mark up output parameters.



Here is the function for response validation:

Function validateXMLResponse (objDictionary)
Set xmlDoc = CreateObject("Msxml2.DOMDocument")
'loadXML
xmlDoc.loadXML(objDictionary.Item("XMLResponse"))
'Check XML syntax
If (xmlDoc.parseError.errorCode <> 0) Then
Set myErr = xmlDoc.parseError
objDictionary.Item("XMLResponseResult") = "Failed"
objDictionary.Item("XMLResponseResultDetails") = "ERROR: " & myErr.reason
Set validateXMLResponse = objDictionary
Exit Function
End If
Set root = xmlDoc.documentElement
'Put all test parameters in array
arrAllKeys = objDictionary.Keys
arrAllValues = objDictionary.Items
'Update values
For i = 0 to Ubound(arrAllKeys)
'Only process #-type parameters
If Left(arrAllKeys(i),1) = "#" Then
strParameterElementName = arrAllKeys(i)
strParameterElementName = Right(strParameterElementName,Len(strParameterElementName)-1)
strParameterElementValueExpected = arrAllValues(i)
'Get element and value
Set node = root.getElementsByTagName(strParameterElementName)
If node.Length > 0 Then
For n = 0 to node.Length-1
'Return actual value in [element]_Out
objDictionary.Item(strParameterElementName & "_Out") = node.Item(n).Text
strParameterElementValueActual = node.Item(n).Text
'Set result
If strParameterElementValueExpected = strParameterElementValueActual AND _
objDictionary.Item("XMLResponseResult") <> "Failed" Then
objDictionary.Item("XMLResponseResult") = "Passed"
Else
objDictionary.Item("XMLResponseResult") = "Failed"
objDictionary.Item("XMLResponseResultDetails") = objDictionary.Item("XMLResponseResultDetails") & _
strParameterElementName & " = " & strParameterElementValueActual & " , expected value " & _
strParameterElementValueExpected & " ;" 'use ; as separator
End If
Next
End If
Set node = Nothing
End If
Next
'Return
Set validateXMLResponse = objDictionary
'Clean up
Set root = Nothing
Set xmlDoc = Nothing
End Function

So finally we have these steps all together:

' Example of XML/Web Service automation in VBScript using WinHttp, XML DOM and ADO
' by Stefan Thelenius march 2008

'Set test case id and data source file
intTestCaseID = 1
strMyDataFile = "c:\MyTestData.xls"

'Get test case parameters
Set objMyTestParameters = getTestParametersIntoDictionary (strMyDataFile, _
"TestCase", "TestCaseID", intTestCaseID)

'Get global parameters
Set objMyGlobalParameters = getTestParametersIntoDictionary (strMyDataFile, _
"Global", "GlobalID", objMyTestParameters.Item("GlobalID"))

'Merge parameters
Set objMyTestParameters = mergeDictionaries (objMyGlobalParameters, objMyTestParameters)

'Create request from template
objMyTestParameters.Item("XMLRequest") = getXMLTemplate (objMyTestParameters.Item("XMLTemplateFile"), _
objMyTestParameters.Item("XMLTemplate"))

'Insert test parameters
objMyTestParameters.Item("XMLRequest") = setTestParametersInXMLRequest (objMyTestParameters)

'Send request
objMyTestParameters.Item("XMLResponse") = sendXMLRequest (objMyTestParameters)

'Validate response
Set objMyTestParameters = validateXMLResponse (objMyTestParameters)

'Display output
Msgbox objMyTestParameters.Item("XMLResponse")
Msgbox objMyTestParameters.Item("XMLResponseResult")
Msgbox objMyTestParameters.Item("XMLResponseResultDetails")
Msgbox objMyTestParameters.Item("ConvertTempResult_Out") 

AdvancedQTP

I am joining forces with the team behind AdvancedQTP as a moderator for some of their forums. The site is an excellent resource regarding QTP and vbScript and I strongly recommend it as first stop knowledge base since HP/Mercury's dito is hard to find these days...

XML automation - Part IV - Sending XML request

This week it is time to send the Web Service request and recieve a response...I have also added some Global parameters in a new sheet in my Excel file so that we can handle test environment properties and so on:



A also added a function for merging Dictionary objects which I use sometimes in my framework. It might not be the best code design approach but I think the concept works well, create a number of dictionaries, merge them together and pass one dictionary to the test case including all test parameters.

Function mergeDictionaries (objDIC1, objDIC2)
If IsObject(objDIC2) Then
    allKeys = objDIC2.Keys
    allItems = objDIC2.Items
    For i = 0 To objDIC2.Count - 1 'Iterate through the array
        If objDIC1.Exists(allKeys(i)) Then
            'Value in objDIC2 is master
            objDIC1.Item(allKeys(i)) = allItems(i)
            Else
            objDIC1.Add allKeys(i),allItems(i)
        End If
    Next
End If
Set mergeDictionaries = objDIC1
End Function

And finally the funcion for sending the request

Function sendXMLRequest (objDictionary)
Dim WinHttpReq
Const HTTPREQUEST_SETCREDENTIALS_FOR_SERVER = 0
Set WinHttpReq = CreateObject("WinHttp.WinHttpRequest.5.1")
WinHttpReq.Open "POST", objDictionary.Item("PostURL"), False
WinHttpReq.setRequestHeader "Content-Type", "text/xml;charset=utf-8"
WinHttpReq.setRequestHeader "Accept", _
"text/xml, multipart/related, text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2"
WinHttpReq.setRequestHeader "Content-Transfer-Encoding", "binary"
WinHttpReq.setRequestHeader "Connection", "keep-alive"
WinHttpReq.setRequestHeader "SOAPAction", objDictionary.Item("SOAPAction")
If objDictionary.Item("UserName") <> "" Then    
    WinHttpReq.SetCredentials objDictionary.Item("UserName"), objDictionary.Item("Password"), _
    HTTPREQUEST_SETCREDENTIALS_FOR_SERVER
End If
'Add WS header and foot if SOAP
If objDictionary.Item("RequestType") = "SOAP" Then
    strHeader = "<?xml version=""1.0"" encoding=""utf-8""?>" &_
        "<soap:Envelope xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance""" &_
        " xmlns:xsd=""http://www.w3.org/2001/XMLSchema""" &_
        " xmlns:soap=""http://schemas.xmlsoap.org/soap/envelope/"">" &_
        "<soap:Body>"
    strFoot =  "</soap:Body></soap:Envelope>"
End If
WinHttpReq.Send(strHeader & objDictionary.Item("XMLRequest") & strFoot)
sendXMLRequest = WinHttpReq.ResponseText
End Function

So all together it should look like this:

intTestCaseID = 1
Set objMyTestParameters = getTestParametersIntoDictionary ("c:\MyTestData.xls", _
"TestCase", "TestCaseID", intTestCaseID)
Set objMyGlobalParameters = getTestParametersIntoDictionary ("c:\MyTestData.xls", _
"Global", "GlobalID", objMyTestParameters.Item("GlobalID"))
Set objMyTestParameters = mergeDictionaries (objMyGlobalParameters, objMyTestParameters)
objMyTestParameters.Item("XMLRequest") = getXMLTemplate (objMyTestParameters.Item("XMLTemplateFile"), _
objMyTestParameters.Item("XMLTemplate"))
objMyTestParameters.Item("XMLRequest") = setTestParametersInXMLRequest (objMyTestParameters)
objMyTestParameters.Item("XMLResponse") = sendXMLRequest (objMyTestParameters)
Msgbox objMyTestParameters.Item("XMLResponse")

Up next: Part V - Validate the response