Introduction

Salomé-TMF is a test management framework. Salomé-TMF offers features for creating and executing tests. We use the concept of tests defined in the norm ISO9646. Tests can be manual or automatic, tests are organized in campaigns and are executed with different datasets in different environments. For making test execution fully automatic, we integrate a script language based on Java in our tool, as one of several plugins which extend Salomé-TMF functionalities.


Contents


Basic concept definitions

Test

A test is the execution of a program or of a sequence of actions, in a given environment. The goal is to check that the tested software fulfills its specifications, by isolating discrepancies between the results obtained and those waited for.

Test suite

A test suite is a logical set of tests.

Test family

A test family is a set of test suites.

Dataset

A dataset is a set of parameters with their values.

Test campaign

A test campaign is a set of tests which will be executed with different datasets in different environments.

Environment

An environment is a set of elements describing the environment in which tests will be executed:

Execution

A campaign execution links three elements: a test campaign, a dataset, and an environment. A campaign execution is launched and its results are kept as an archive and can be consulted.


Organisation and test description

The way to describe tests is willfully very directed in the Salomé-TMF test management framework, following the concepts of the norm ISO 9646. Test are organized first by family, then by suite, a suite comprising tests as "atomic" elements (Figure [*]).

Figure: Test plan

Image testplan

Tests can be of one of two categories: either automatic or manual. A manual test is made of the description of several steps which need to be executed. For each step, a verification to be made by the tester is included.

An automatic test comprises a script or a test program which will be executed by the Salomé-TMF tool, which will automatically kept the result in the corresponding test campaign execution.


Creating test campaigns

When the set of tests is described in the tool, the user can define test campaigns (Figure [*]). A test campaign is a set of tests, defined indepently of the test suites and families. A test campaign will be executed in a given test environment, using given datasets.

Figure: Test campaign

Image campplan


Test execution

Test campaigns are defined independently of the datasets and the execution environments. For launching the execution of a test campaign, you need to link a campaign with an execution environment and a dataset.

Those notions of environment and datasets enable to launch the tests for a given campaign on several versions of test environments and datasets in a simple way.

An environment is the target for a test campaign execution. It is made of a description, an initialization script, and a set of valued parameters which can be used in the tests. This dataset must give a value to all the parameters used in at least one test of the campaign. A test campaign execution, or shortly an execution (Figure [*]) is defined as the linking between those three elements: a campaign, an environment and a dataset. When defined, an execution can be launched one or several times and the test results or verdicts will be attached to the execution for every launch.

Figure: test campaign execution

Image execWindows


About this document ...

This document was generated using the LaTeX2HTML translator Version 2002-2-1 (1.71)

Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999, Ross Moore, Mathematics Department, Macquarie University, Sydney.

The command line arguments were:
latex2html -split 1 -no_navigation -prefix c1 -dir ../.././src/docs/html/en -no_footnode en/introduction.tex

The translation was initiated by on 2006-03-10


2006-03-10