|  |
Combinatorial Test Services
A software library for generation and manipulation of testing input data or configurations.
Date Posted: January 12, 2004
|
|
|
|
|
Update: March 3, 2004
Version 1.1: Bug fixes. Support added for g++ compiler Version 3.3.1 under Linux; capitalization problems fixed in "include" file names; name of pair class changed to avoid conflicts; "make" files added to the sample.
What is Combinatorial Test Services?
The Combinatorial Test Services (CTS) is a software library for generation and manipulation of testing input data or configurations. CTS enables the user to generate small test suites with strong coverage properties, choose regression suites, and perform other useful operations for the creation of systematic software test plans.
CTS defines the basic classes needed for manipulating combinatorial test cases, test suites, and, testing parameters. These classes then provide a wide variety of services needed by the testing community, including the following:
- Creation of test suites with a given level of coverage
- Creation of test suites of a given maximum size
- Reduction of test suites to manageable regression sets
- Extending an existing test suite to provide a desired level of coverage
- Removal of test cases that contain forbidden combinations
- Construction of test suites without any forbidden combinations
- Printing the test suites in a format suitable for the use of a test execution engine
- Printing the test suites in a format suitable for import into MS Excel
How does it work? As an example, consider the testing of an Internet site that must function correctly on three operating systems (Windows®, Linux®, and Solaris), two browsers (Explorer and Netscape), three printers (Epson, HP, and IBM), and two communication protocols (Token Ring and Ethernet). Although there are 36 (=3X2X3X2) possible test configurations, the nine tests in Figure 1 cover all the interactions between different pairs of parameters of the system.
The interactions between operating systems and printers are all covered precisely once, but some interactions between operating systems and browsers are covered more than once. For example, Windows and Explorer are tested together twice in the test suite.
| Operating System |
Browser |
Printer |
Protocol |
| Windows |
Explorer |
Epson |
Token Ring |
| Windows |
Netscape |
HP |
Ethernet |
| Windows |
Explorer |
IBM |
Ethernet |
| Linux |
Netscape |
Epson |
Token Ring |
| Linux |
Explorer |
HP |
Ethernet |
| Linux |
Netscape |
IBM |
Token Ring |
| Solaris |
Explorer |
Epson |
Ethernet |
| Solaris |
Netscape |
HP |
Token Ring |
| Solaris |
Explorer |
IBM |
Ethernet |
|
|
|
| | About the technology author(s):
Alan Hartman holds a Ph.D. in mathematics from the University of Newcastle, Australia; an M.Sc. in mathematics from the Technion, Israel Institute of Technology; and a B.Sc. (Hons) in mathematics from Monash University in Australia. He joined IBM Haifa Research Laboratory in 1983 after a postdoctoral fellowship at the University of Waterloo in Canada. He has spent two periods on leave at the University of Toronto and at Telstra Research Laboratories. His career at IBM has included research in storage technologies, communications network design, mathematical optimization, and, most recently, hardware and software verification. Dr. Hartman is currently coordinating the EC Fifth Framework AGEDIS project and is responsible for developing technologies for model-based testing within IBM. Dr. Hartman has published over 50 research papers and holds nine patents.
Leonid Raskin is an M.Sc. student in computer science at the Technion, Israel Institute of Technology. He holds B.Sc. (Hons) in computer engineering from the Technion, Israel Institute of Technology. Mr. Raskin joined IBM Haifa Research Laboratory in 2002. |
|
|
| |
|
