Say Uncle … how to script with an automated testing tools and also if you could give me some good practices pleaaaaaase?

5 02 2009

Hum … and you do not want to 100 dollars and a chocolate? In short, I feel you are impatient  when listening so…

I will try to explain how to automate the functional test. I will talk about QuickTest Pro (QTP) and other selenium. Robots used for functional testing, test scenarios and test users.

There are two ways to script in QTP, it is good … and good!
Let me explain, as you might think that I do make a quick and sterile wink sketch of the “inconnu” (to our English friends: group of French comic of 90 years).

But before reviewing the ways of scripting ask what we can serve these famous robots that could be the miracle solution to testing (or not …).

Automata theory tests:
On a website (for example) QTP records your actions (recording of mouse movement, mouse clicks, etc.). And plays and replay at will. You can execute an infinite number of scenarios to test all the evening and the morning after your coffee in peace consult performance reports.
This definition is an urban myth spread by the publishers a few years ago when the automated tests were not known to clients. Since then and until now, unfortunately, it is always the thought that most customers…

Other important principles of automated tests:

  • Automatic test function by recognition of GUI objects (buttons, dropdowns, etc.). It stores in a directory called sometimes « repository »
  • During registration, they capture the interactions carried out on items needed during the course of the test. It generates a VB script (for QTP)
  • During implementation, they replay the test from the script

Automatic testing in real life:
The only true part in the theory definition is « QTP records your actions and will play it!
The rest is only possible with very little customer with a significant level of maturity in the automation of tests. In short, there are two ways to script:

The scripting for recording and reworking:
– The tester has the GUI and script directly
– It runs the script identifies and corrects errors
– Once the script is complete it must be validated by a tester and make a cross-checking (very important to finalize the scripts)

Scripting mode programming descriptive and reworking:
– The tester has no GUI and must work from technical and functional specifications. But this is very rare among customers. Furthermore this means that the specifications are complete, validated and stable… (it is good to laugh sometimes)
– It runs the script identifies and corrects errors
– Cross-checking

Important points to consider:
Principle: We automate the testing for application on a MATURE and STABLE application! Otherwise, the loss of time and masturbate to Mammoth (sorry but this is the central idea)!

– Principle: 100% of manual tests are not automatable:

  • You should know prioritize, make smart choices, use of the law of Paretto (80 / 20)
  • Explore the benefits of automation, etc.

– Principle: a feasibility study to determine the value of automated tests and technical feasibility (see another article in this subject)
– Return on investment (ROI): the automated test campaigns are profitable from only three or four iterations of automatic test campaign (from a curve ROI over three years)
– Creating scripts:

  • the burden of creating scripts is more important than the creation of manual tests
  • The difficulty is not in the capture of the script but in the choice of criteria and means of control of outcome
  • there must be synchronization points or between two actions to avoid times out on the occurrence of outcome
  • Good practice for a single scenario (better robustness of the scenario = recover under the same conditions as when you save the script):

– Preparation and application environments
– Generation of tests data
– Testing the application
– Cleaning and application environments
– Test data:

  • think about generating tests data by the automate before the execution scenarios

– Execution of scripts: load auto run is less important than manual execution
– Maintenance script: good methodology and good practices are to be applied before the start of automation to minimize the burden of maintenance scripts (standards for the organization, naming, version management scripts, etc.).
– Maintenance of scripts: it is necessary to maintain competence of automation to make the adaptations and changes in scripts if developments in the application

I finish by asking you to ALWAYS coupler and where possible automate your repository (tests management tool) with your test! This will:
– Store the description of scenarios to automate
– Store the QTP scripts in QC and associated repository
– Planning for the implementation of automated test campaigns
– Store the results of test campaigns

Say Uncle … how to do a feasibility study prior to automate my tests?

5 02 2009

Principle: a feasibility study to determine the value of automating tests and technical feasibility. Below the summary type of a study, with some guidance to help you complete this document.
1 – First of all we must validate the technical aspects of automation: ensuring that the controller is suitable for a technical point of view to the automation of the application (on a sample of the batch scripts to automate)

  • Using documents prerequisite automata

2 – Speak the GO / NOGO

3 – Make Prototyping:
– Identify scenarios representative scenarios using existing manuals:

  • Scenario Complexity Simple
  • Scenario complexity « Medium »
  • Scenario complexity « Complex »

– Setting up scenarios and Quality Center /TESTLINK / Salome QuickTest Pro / SELENIUM:

  • Automation scenarios (eg selection criteria scenarios):
  • Scope: to automate functions stable (test of non-regression tests and no automatic changes)
  • Frequency: automate testing of functions to be tested regularly
  • Criticality: automate critical functions
  • Estimate the duration of automation for each scenario
  • Estimate the manual execution scenarios (to compare the charges)
  • Estimate the cost of maintenance scenarios there is a version

– Establish the beginnings of ROI:
– Identify « Best Practices » to implement

  • Library functions of the application, VB reusable functions, object management, etc..
  • Refer to the article on how to automate and good practices

– Identify the method for managing data sets
– Make recommendations on the management of versions of the scripts.

Below a small example of a feasibility report. This document covers:
Study Objectives
Client_xxx and your_company we agreed to conduct a feasibility study to determine if an automated test campaigns is possible.
This could yield savings in time result. Reusability as well as by third parties other than the teams your_company.

The objective of this study therefore is:
– What are the various modules / parts automatable?
– With what resources (software, tools) to automate these tests?
– The gains are significant?

  • How did we do?

A first choice initially focused on the controller (QuickTest Pro / SELENIUM) in view of the constraints and needs Client_xxx.
Then, from the scenarios in (Quality Center / TESTLINK / Salome)  we selected a few tests as significant.
Then we « Recorded and scripted » using the controller.
To illustrate more clearly the possible explanations: the video below shows an example of test automation application for AAA, with the automatic yyyy (name of the feature).
The software automatically tests:

  • Step 1
  • Step n

Attach the video of the action of the automaton (the tools available on the Internet and free).
It is important to take a concrete example or the impact on the customer will be weakened and less percussive.

  • Results of the study

The feasibility study has shown that:
– Most applications are compatible with the automation tool xxx. This compatibility has been verified through the document « ddd », provided in the appendix.
– A lot of the scenarios are automatable by Selenium:
The ‘mmm1 « , which is to verify the proper functioning of xxx
The ‘mmm2
Parts not automatable affect trade flows between different modules:
Exchange (the problem of detecting objects by xxx), etc.

Preview: new versions of Quality Center and QuickTest Pro V 10.00!

5 02 2009

I attended the presentations of new versions of Quality Center (QC) and QuickTest Pro (QTP) in the version 10.00. This HP in Issy les Moulineaux (Near to Paris – France) a rainy morning. And so, in the TGV (a very fast railway for the English) brings me back, I type this article to share with you the information provided by HP in their basement …
The celebration program was:

  • New QC 10.00
  • New BPT 10.00
  • New QTP 10.00
  • Method of pricing (on the licensing policy – a subject not treated here)
  • Demos

(Does the use of figures 10.00 warns that there will be many patch and it is expected to version 10.94 Only time will tell).
HP has reported that all the information and risk are subject to because not included in versions 10.00 in February. Yet I take the bet that if. This warning of HP is probably the only legal title.
I need not address in this article from Business Process T. (BPT) for the moment because it’s uncommon among French customers and above all a desire / wishes for HP to sell full containers (first desire of any software publisher after all).

Quality Center (QC) 10.00:
Regarding what Quality Center is, I refer you to my articles on tools for testing repositories / management as well as those on QC. The release date is scheduled for 31 January 09 and three packs will be distributed:

  • QC Starter Edition: Designed for a team of 5 single-project
  • Enterprise Edition QC: QC 9.2 equivalent current (TD for Quality Center)
  • First Edition: Designed for a team with needs such as advanced multi-project, high availability platform supporting QC, etc.

I will detail here only the Enterprise Edition / First of QC because it is where the real changes / new features.
QC Enterprise Edition / First, the modules present:

  • Requirements management
  • Test plan
  • Lab Tests
  • Risk based testing
  • Release management
  • Defect management
  • QA Lab Manager

The sentence shock input presentation by HP was « Mastering the chaos version requirements, tests and components”! Rather ambitious and shock, but see news (grouped by topic below).

Centralizing these unlike the current dashboard (9.2)

  • New module replacing the dashboard
  • Centralized reporting
  • All panels are centralized in the module
  • Creation of public view and private
  • Report possible multi-project
  • Take this version of reporting as the V1 from the old … Finalized dashboard module so.

Possibility to define a more complete test:

  • Risks and technical occupations ==> assigned to a functional complex – calculation of risk

Versioning: this module is not useful for a small project
Version Management requirements, tests (test sheets for example) and components:

  • Check-in and check-out
  • Comparison between the
  • Ability to return to a previous version
  • Can be activated at a project through the hotel (the new business administrator, therefore)
  • The versioning is of course to be taken into account before the project starts (part of best practices dear to CMMi and other ITIL)
  • Establishment of baselines (creating a picture of a set of project components), to mark / identify the milestones of the project. Example: requirements validated V1: establishing a baseline. It also allows you to freeze a configuration of testing association with a test campaign.
  • Guarantees to run the correct version vis-à-vis a test campaign
  • Comparison of baseline possible ==> identifying the elements that have changed

QC Requirements Management:

  • It would be a direct competitor tools like Doors, Req Pro and other Caliber
  • HP cites a 2008 report issued by Forrester and noting that the new management module requirements of HP is the first before all others. Hmm … I just think this kind of premature conclusion since the version that comes out in February and that results can be obtained after a few months of intense use in the real customers. A monitor then.

Other: template:

  • It is now (finally?) possible to define a project template and especially to use it and share it between many projects. The major fact is the possibility to update the template and distribute the update on the entire project implementing it. You should know that the current version 9.2 is only possible through internal development of each client.
  • Ability to setup and maintenance of centralized template
  • Broadcast « automatic » updates
  • Maintenance of template by administrators
  • Please note that this feature also joined the concepts of best practices advocated by CMMI and ITIL

Functional testing:

  • Output January 31
  • WinRunner is no longer included in this pack

Contained in the pack Functional testing:

  • Resource Management QTP:
  • Versioning
  • Tools Comparison

QuickTest Pro (QTP) 10.00:
Better reporting

  • Monitoring local system: this module allows RUN when scripts to monitor the implementation and some points. Specifically we can identify at what point and following the action script a problem occurs. For example, a peak CPU load generated by the action of a QTP script. This will help to strengthen links and interactions between the teams test and those in charge of performance tests. It should be noted that this function has been implemented by many customers through internal development. They will appreciate.
  • Improvement of the various GUI:
  • Tools management tasks (TODO)
  • Support for scripting
  • Export to DOC format and PDF reporting
  • Ability to attach screenshots
  • Direct access to the script from QTP reporting
  • Ability to compare two versions of scripts easily

My final impression is that QTP is transformed into CASE (Computer Aided Software Environment or AGL (Atelier de Genie Logiciel for French people)) but oriented tests. What I mean is that HP has incorporated many features to better script and more to help developers fully QTP scripts. I find it beneficial for all writers. Do not draw hasty conclusions and wait till the first returns of experience other than HP. Finally, synergy, linkages between QTP and QC seem also strengthened.

What about migration?
To migrate your versions to 10.00 you must already have version 9.0 (patch 26) and QC 9.2 (patch 12). Unable to migrate with older versions as these.
An upgrade tool is available on the HP site to help you with migration. But be careful if this works as the previous times you need to go through testing experts to help you because the migration tool from HP does not expect miracles and calls for the intervention of humans.
One last thing about the migration, if you go to QC 10.00 you should necessarily go to the 10.00 version of QTP.

Do we collect on the body of this bloodless automation testing tool that was a forerunner in our industry. He well-deserved retirement. Rest In Peace WinRunner and thank you for everything.

  • It is no longer distributed with the pack functional testing
  • End 2009: End of the developments and patch
  • End 2011: The end

Goodies and “petits fours” (little cakes for corporate party in France):
And the goodies offered by HP for us to reward our time and attention? Well we had a pink plastic pen, a few blank sheets to the header of HP … and a book in English on the tests!
His name is « Optimize Quality for Business Outcomes – a practical approach to software testing » and this in its third edition. This is the first time I have heard of this book dealing with the tests. Below I list the main chapters:
1. What is the big deal about testing?
2. Testing the business requirements: start at the root of the problem
3. Test rules: build the backbone for effective testing
4. Test cases: let’s get down to the real stuff
5. Test optimization: balancing risk and effort
6. Why bother with non-functional testing?
7. Application security testing: the next frontier
8. Test sourcing: how outsourcing improves cost effective testing
9. Successful goal-driven approach KPI
10. Getting started: putting it all together
11. Appendix A – Common test techniques
12. Appendix B – HP application quality management solutions introduction and overview
13. Appendix C – Verification
14. Appendix D – Naming conventions

I try to read this book quickly and I will make my opinion, soon.

And then?
Ultimately, HP’s strategy is to play together in the editing software to expand its functional and its turnover. What is a healthy and able to provide end users with better service and services companies expanding their services (with a contribution of value added greater than today).
However, HP blithely tramples the toes of many competitors / software editor. So publishers with tools for managing requirements and configuration management / version will inevitably their market shares reduced inexorably by HP. As inexorable as desertification in Africa? Not so on this because, take for example publishers management software requirements, their products will not be dedicated more complete than that of HP. It remains to be seen more fully the choice of HP doing in this area and the effort to test them in real life!

Criteria for selecting a management testing tool (or quality management tool)

5 02 2009

Criteria for selecting a management testing tool (or quality management tool):

This article will help you to choose a management testing tool in peace. Why? Because for some client it’s not easy as we can believe. Example: a client already has a tool from HP (Quality Center) and at the same time using Enterprise Architect (EA) to manage its requirements. The question is « Is what we use quality center or EA? ». From this question there was a stir on the choice. And inter-service policy and the brake of some users does not help the stand-by was delivered … until the list below is made.

This list exists only to provide FACTUAL information about the main functions to be performed by a management testing tool. As you may be able to see it is not oriented for Quality Center is chosen (I want to say it’s not irony).

A repository of tests should be able to manage (only basic functions):

Level 1 Functions
Manage the test requirements:
* Support for the creation
* Multi-user: easy sharing between all project stakeholders
* Easy change
* Easy link between requirements> Test Sheets > Defects
* Generating a matrix coverage requirements ==> campaign / scenarios
* Analysis of coverage
* Print friendly list
* Version management requirements

Manage the test plan to test cases:
* Support for the creation
* Easy change
* Ability to attach documents
* Multi-user
* Version management

Manage books / files / test plan:
* Support for the creation of scenarios
* Multi-user
* Version management

Help manage tests:
* Status of design tests
* Status of the test run
* Status of an overall campaign test
* Printing of reports already formatted and filled
* Multi-user

Function Level 2:
Manage defects:
* Tools for creating worflow defects
* Create and complete an information sheet defect
* Possibility of attachments, screen captures, etc.
* Integrated search engine
* Easy and direct link between a sheet / test cases and an defect
* Notification by mail to the defects involved in the project
* Multi-user
* Printing of reports already formatted and filled with figures and statistics modules / functions, etc.

Interfacing and direct steering:
* Automates functional tests
* Automates testing techniques (performance)
* Defects (if not integrated)