Efficiency of Software Testing Techniques: A Controlled Experiment Replication and Network Meta-analysis
Journal Title: e-Informatica Software Engineering Journal - Year 2017, Vol 11, Issue 1
Abstract
Background. Common approaches to software verification include static testing techniques, such as code reading, and dynamic testing techniques, such as black-box and white-box testing. Objective. With the aim of gaining a better understanding of software testing techniques, a controlled experiment replication and the synthesis of previous experiments which examine the efficiency of code reading, black-box and white-box testing techniques were conducted. Method. The replication reported here is composed of four experiments in which instrumented programs were used. Participants randomly applied one of the techniques to one of the instrumented programs. The outcomes were synthesized with seven experiments using the method of network meta-analysis (NMA). Results. No significant differences in the efficiency of the techniques were observed. However, it was discovered the instrumented programs had a significant effect on the efficiency. The NMA results suggest that the black-box and white-box techniques behave alike; and the efficiency of code reading seems to be sensitive to other factors. Conclusion. Taking into account these findings, the Authors suggest that prior to carrying out software verification activities, software engineers should have a clear understanding of the software product to be verified; they can apply either black-box or white-box testing techniques as they yield similar defect detection rates.
Authors and Affiliations
Omar Gómez, Karen Cortés-Verdín, César Pardo
Keywords
Related Articles
- EP ID EP200433
- DOI 10.5277/e-Inf170104
- Views 105
- Downloads 0
Are We Working Well with Others? How the Multi Team Systems Impact Software Quality
Background: There are many studies on software development teams, but few about the interactions between teams. Current findings suggest that these multi-team systems may have a significant impact on software development...
Reducing the Number of Higher-order Mutants with the Aid of Data Flow
Higher-order mutants are created by injecting two or more mutations into the original program, while first-order mutants are generated by seeding single faults in the original program. Mutant generation is a key stage of...
On Visual Assessment of Software Quality
Development and maintenance of understandable and modifiable software is very challenging. Good system design and implementation requires strict discipline. The architecture of a project can sometimes be exceptionally di...
Using the Cognitive Walkthrough Method in Software Process Improvement
In the past years, efforts in the field of Software Process Improvement were increasingly focusing on human aspects making one aware that people participating in the processes have a high impact on the success of any imp...
Experience Report: Introducing Kanban into Automotive Software Project
The boundaries between traditional and agile approach methods are disappearing. A significant number of software projects require a continuous implementation of tasks without dividing them into sprints or strict project...