Combinatorial testing in software projects
by Mario Brčić
Co-authored with Damir Kalpic
MIPRO 2012 - Jubilee 35th International Convention Proceedings / Biljanović, Petar (ur.). - Rijeka : Croatian Society for Information and Communication Technology, Electronics and Microelectronics - MIPRO , 2012. 1832-1837 (ISBN: 978-953-233-069-4).
Software systems continuously grow in size and code complexity, the latter most evident through greater component... more Software systems continuously grow in size and code complexity, the latter most evident through greater component interconnectedness. This leaves more space for bugs which introduce risks such as exposure to security threats. Combinatorial testing looks for interaction failures in order to improve the system security and effectiveness guarantees. One of the most effective test selection approaches under combinatorial testing are experimental design extensions for software testing. Covering array test sets are compact while maintaining at the same time complete combinatorial coverage up to the desired level. Smaller test sets with customizable level of assurance can drive testing costs down substantially. The paper presents a survey of research into combinatorial testing suite factors while also identifying possible future research ideas.
1 views
Seen by:
Download (.pdf) 
Contemporary Trends in Defect Prevention: A Survey Report
by Sami Ulhaq
Muhammad Faizan, Muhammad Naeem Ahmed Khan & Sami Ulhaq
Most of the software projects fail to meet the desired level of quality and standards due to different types of... more
Most of the software projects fail to meet the desired level of quality and standards due to different types of defects introduced during the course of requirement solicitation, designing and development. These defects inexorably hinder the secure deployment or smooth operations of the software systems. One of the key reasons for this misfortune is the lack of proper defect prevention planning while formulating the software architecture. Defect prevention needs to be a thorough and critical phase because it has a direct impact on quality of
the product which cannot be compromised. This paper looks into different defect prevention techniques and analyses them critically. The scope of this study is restricted to the identification of the modern trends in defect prevention.
Towards the Integration of Visual and Formal Models for GUI Testing
by Ana Paiva
This paper presents an approach to diminish the e ort required in GUI modelling and test coverage analysis
within... more
This paper presents an approach to diminish the e ort required in GUI modelling and test coverage analysis
within a model-based GUI testing process. A familiar visual notation a subset of UML with minor extensions
is used to model the structure, behaviour and usage of GUIs at a high level of abstraction and to describe test
adequacy criteria. The GUI visual model is translated automatically to a model-based formal speci cation
language (e.g., Spec]), hiding formal details from the testers. Then, additional behaviour may be added to
the formal model to be used as a test oracle. The adequacy of the test cases generated automatically from
the formal model is accessed based on the structural coverage of the UML behavioural diagrams.
Reverse engineering of GUI models for testing
by Ana Paiva
The incorrect behaviour of Graphical User Interfaces (GUIs) can compromise the effective use of the overall software... more The incorrect behaviour of Graphical User Interfaces (GUIs) can compromise the effective use of the overall software application. One way to discover defects and increase the quality of GUIs is through testing. Test cases can be created manually or produced automatically from a model of the GUI. The size and complexity of GUIs makes it unpractical to do extensive manual testing. However, creating a model of the GUI in order to generate automatically test cases is also a laborious task. This paper presents a reverse engineering approach for diminishing the effort required for constructing the model of an existing GUI. The GUI is exercised by a combination of manual and automatic exploration, and information about its structure and some of its behaviour is automatically extracted, resulting in an incomplete GUI model. This model is subsequently completed manually, validated and used as input for automatic test generation and execution.
A model-to-implementation mapping tool for automated model-based GUI testing
by Ana Paiva
This paper presents extensions to Spec Explorer to automate
the testing of software applications through their... more
This paper presents extensions to Spec Explorer to automate
the testing of software applications through their GUIs based on a formal speci cation in Spec sharp. Spec Explorer, a tool developed at Microsoft Research, already supports automatic generation and execution of test cases for API testing, but requires that the actions described in the model are bound to methods in a .Net assembly. The tool described in this paper
extends Spec Explorer to automate GUI testing: it adds the capability to gather information about the physical GUI objects that are the target of the user actions described in the model; and it automatically generates a .Net assembly with methods that simulate those actions upon the GUI application under test. The GUI modelling and the overall test process supported by these tools are described. The approach is illustrated with
the Notepad application.
Test Coverage Analysis of UML State Machines
by Ana Paiva
Software testing is a very important activity of the software development process. To expedite the testing process and... more Software testing is a very important activity of the software development process. To expedite the testing process and improve the quality of the tests, models are increasingly used as a basis to derive test cases automatically - a technique known as model-based testing (MBT). Given a system model and a test suite derived automatically from the model or created by other process, the coverage of the model achieved by the test suite is important to assess the quality and completeness of the test suite early in the software development process. This paper presents a novel tool that shows visually the coverage achieved by a test suite on a UML state machine model. The tool receives as input a UML state machine model represented in XMI and a test suite represented in a XML format, and produces a colored UML state machine model that shows the coverage result. Model test coverage is determined by simulating the execution of the test suite over the model. An example is presented in order to show the features of the tool.
Testing aspect-oriented programs
by Ana Paiva
This work addresses software quality issues
introduced by Aspect-Oriented Programming (AOP). AOP aims
at... more
This work addresses software quality issues
introduced by Aspect-Oriented Programming (AOP). AOP aims
at enhancing modularity and thus solving the problem of
crosscutting concerns by capturing them into new units of
modularity called aspects. With the increasing usage and
acceptance of AOP, the task of assuring aspect-oriented systems’
correctness has become a challenge, mainly due to its nature.
Although several testing techniques have been applied and
improved for object-oriented programs through the years, there
is still the problem of selecting and verifying which ones can be
effectively applied to AOP. This paper presents a perspective on
how AOP can improve quality (when applied correctly),
identifies a set of key testing issues that arise with AOP and
presents weaknesses and strengths of the existing solutions to
improve quality in AOP.
Specification-based Testing of User Interfaces
by Ana Paiva
It is proposed an approach to integrate formal methods in the software
development process, with an emphasis on... more
It is proposed an approach to integrate formal methods in the software
development process, with an emphasis on the user interface development.
The approach covers the specification by means of formal models, early model
animation and validation, construction and conformity testing of the user interface
implementation with respect to the specification. These conformity tests
are described in detail through a state transition model with an abstraction function
mapping concrete (implementation) to abstract (specification) states and
operations. In order to illustrate the approach, it is presented a simple
login/password dialog specification in VDM++, using a reusable control specification
library, with a straightforward translation to Java or C#.
Specification-driven Unit Test Generation for Java Generic Classes
by Ana Paiva
Several approaches exist to automatically derive test cases that check the conformance of the implementation of... more
Several approaches exist to automatically derive test cases that check the conformance of the implementation of abstract data types (ADTs) with respect to their specification. However, they lack support for the testing of implementations of ADTs defined by generic classes. In this paper, we present a novel technique
to automatically derive, from specifications, unit test cases for Java generic classes that, in addition to the usual testing data, encompass implementations for the type parameters. The proposed technique relies on the use of Alloy Analyzer to find model instances for each test goal. JUnit test cases and Java implementations of the parameters are extracted from these model instances.
Test Coverage Analysis of UML Activity Diagrams for Interactive Systems
by Ana Paiva
User interface testing is a very important but time
consuming activity. To automate and systematize the... more
User interface testing is a very important but time
consuming activity. To automate and systematize the testing
process, models can be used to derive test cases automatically – a
technique known as model-based testing. Given a model
representing the intended system behavior and a test suite
derived from the model or produced manually, the coverage of
the test suite over the model is an important early indicator of the
quality and completeness of the test suite. This paper presents a
novel tool that shows visually the coverage achieved by a test
suite over an UML model of an interactive system. This model is
based on activity and class diagrams, with special user interface
modeling features (stereotypes and keywords) inspired in
ConcurTaskTrees and Canonical Abstract Prototypes. The tool
receives a UML model file and a test suite, determines the model
coverage by simulating the execution of the test suite over the
model, and produces a colored UML model showing the elements
covered. An example is presented to illustrate the features of the
tool.
Reverse Engineered Formal Models for GUI Testing
by Ana Paiva
This paper describes a process to reverse engineer structural
and behavioural formal models of a GUI application... more
This paper describes a process to reverse engineer structural
and behavioural formal models of a GUI application by a dynamic
technique, mixing manual with automatic exploration. The goal is to
diminish the effort required to construct the model and mapping
information needed in a model-based GUI testing process. A skeleton
of a state machine model of the GUI, represented in a formal pre/post
specification language, is generated automatically by the exploration
process. Mapping information between the model and the
implementation is also generated along the way. The model extracted
automatically is then completed manually in order to get an executable
model which can be used as a test oracle. Abstract test cases, including
expected outputs, can be generated automatically from the final model
and executed over the GUI application, using the mapping information
generated during the exploration process.