Awareness to the testing

27 09 2008

The Software Engineering or GL is a relatively recent: the first conference on GL took place in 1969 during the NATO conference. Besides the following quotation is taken: « The software engineering is the development and use of engineering principles to economically produce reliable software, and operate effectively on machines real » Naur & Randell ( eds), Software Engineering: A Report on a Conference sponsored by NATO Science Committee, NATO, 1969.

So, the software engineering touches the life cycle of software. All phases of creating software are taught: analysis of needs / requirements, development of specifications (functional and technical), development, testing phase and finally “production”. The inclusion of tests following a methodology (admittedly craft) and the quality of software began at that time.

Some definitions:
• « Practical application of scientific knowledge in the design and development of software and related documentation required to develop, implement and maintain » (BW Boehm, 1976),
• « All the design and implementation of products and processes to streamline production software and its follow-up » (Order of 30-12-83).

DEFINITIONS: QUALITY TESTS

“The quality of software is its ability to respect the needs / requirements (expressed or potential) users (Martin, JP, 1987). “

Software development is a human activity introduces defects at all stages of the process and especially when the activities of testing and verification, defects propagate, produce and other defects are delivered to customers. Software development is also a massive collective and involves complex issues such as sharing of tasks that requires an adjustment between different people and points of view or the compression of time and the imperfection of communication, geographical distance does not the adjustment, the likelihood of introduction of defects increases. (Bug from “Mars exploration”: not the same system of measurement).

The software development is an iterative and written communication plays a major role, but the size and number of documents does not allow individual ownership. Desynchronization of the updated reference documents in relation to decisions can generate anomalies, as well as the lack of synchronization is the source of many errors especially at interfaces. Finally, journals and tests are ways resynchronization references.

The aggravating factors are the size and complexity of applications and the size of the teams. But also, changing technologies and the pressure of time or geographical distribution with teams spread over various continents. And therefore a clash of culture and language.

As you can see the software is complex and can generate at all stages of anomalies.

Increase the rate of errors in production for 1000 lines of source code

The illustration above is an example of progress in the rate of errors (in production) to 1000 lines of code developed. The more line increases, the mistakes are numerous.

Some sources:
• an experienced programmer injects 1 defect every 10 lines of code or 100 defects per KLOC (kilo line of code)
• after integration and validation, it is 5 to 10 defects per KLOC (Source: Bugs or defects Watts S. Humphrey SEI Interactive, March 1999).

The audit software is designed to demonstrate that software products from a phase of the development cycle conform to specifications and requirements (including legal and regulatory requirements) established in previous phases.
It also aims to detect and report misconduct that may have been introduced in phases before the audit.

The tests are used to detect differences between the expected and observed behavior during testing, which eliminates many errors in the software.
The tests also make sure the quality level of an application.
Finally they can gain the confidence necessary before operational use (it should be noted that the number of detected faults can not be considered as a criterion for success of the tests. The feedback shows that indeed complex technical and industrial constant, a large number of errors compared to other projects « reference » can only be interpreted as an indicator of a software containing a large number of mistakes and not the achievement of a good detection rate of these mistakes).

It is very difficult to have confidence in software with a large number of faults detected by the test.
There is a risk for the designer to be defined by a single entity (individual, team), the specification, design, test strategy and test cases. This implies «  we can not be judge and jury« .
The meaning of this expression in the middle of testing expresses the fact that a team of developers is not the most likely to provide application development and test. At least two reasons justify the recourse to a third party to test software. The first reason is that the goal of testing is to run a program with the intention of finding mistakes, which is a mental process unnatural for its author, difficult to be carried out by a single entity.
The second reason is that the program may contain errors due to “misunderstanding” of the implementation or specifications by the developer. In this case, it is likely that it will be the same «  misunderstanding  » when he tested his own program (common mode of failure).

The correction of errors software can inject new mistakes and disturb the correct parties already tested, but also make active part of the software previously inaccessible and therefore reveal a large number of new faults, which could not be found before this correction.

Now that the books say?
« The test is not an exact science. It uses common sense as well as high technology « (Maurice Rozenberg – Tests Software).

« The test is implementation or evaluation of a system or component, through automatic or manual to see if it meets its specifications or identify differences between the outcomes and results » (version of the IEEE).

« The test is a technical inspection to ensure through its execution, that the behavior of a program complies with pre-established » (according AFCIQ).

These official definitions of what testing is all true but are also complementary. Each of these definitions reviews with a specific qualification activities taking place within a qualification project.

Maurice Rozenberg explains that the test must combine know-how and technological knowledge. The IEEE and AFCIQ address the topic by posing as a fundamental test (equipped or not) is the measure of the difference between the specifications of users (expressions of needs or equivalent) and results.

In fact, the test phase / qualifications / recipe (“recette” is the French professional term in testing!) is an integral part in a global project related to an application (creation, evolution, correction) that take place in the ecosystem for one or more . To do this we must take sufficient decline.

“Think globally, act locally!”

This belief must be at the forefront of all project leaders. Because, to believe “testing begins only after the developments “ is already a serious mistake and mortgage sharply the following of the qualification project.

Finally, for quote the Software Engineering: « Say a software package that is high-quality means we can apply certain criteria as to the adequacy at the requirements, reliability, efficiency and scalability. »

In practice, it aims to test the software using a methodology with adequate tools and robots.

The other faces of the quality is none-quality. This will be addressed in the next article.