Test process optimisation through big data analysis
Publication Date
July 22, 2023
Creator
Abstract
The goal of the research is to adopt artificial intelligence techniques for regression test case selection in an industrial setting. Currently, the selection of test cases is made manually by test engineers referring to software change documentation. A machine learning-based solution which selects test cases with high potential in uncovering software defects is proposed and results were analysed.
Past historical results of test cases and various metadata, such as the ID, name and priority of test cases, were used in data training. At the pre-processing stage, data were analysed, cleaned and normalised. Then, a two-part balancing method, comprising of outliers removal and resampling with an algorithm, was applied to the imbalanced data before it can be fitted to various machine learning models.
The model that best fits the system requirement, which recommends at most 50% of the total test cases with no false negative predictions and as few false positive predictions as possible, is selected to be implemented as an executable application. The finalised model, based on random forest, recommends 1,626 test cases (7.35% of total test cases) for execution with no false negative and 882 false positive predictions (3.98% of total test cases) out of 22,137 test cases. This fulfils the two objectives of this research, which is to construct a recommendation system capable of reducing at least 50% of all test cases, and to ensure that there are no false negative and minimal false positive predictions.
The current test system is a black-box system, meaning the software’s functionalities are tested without accessing its internal code structure. The model’s performance can be improved with a white-box test system, where the software source code can be accessed, and information such as which specific code segments are causing test case failure is available.
Item Type
ethesis
Thesis Type
MPhil
Supervisors
Subjects (LC)
Associated Schools / Departments
Department of Electrical and Electronic Engineering
eprints ID
74011
UoN Repository URI
Except where otherwise noted, this item's license is described as
File(s)![Thumbnail Image]()
Name
MPhil Thesis - Kho Xiang Juan (20220157).pdf
Type
Full-text
Description
Reader
Size
2.16 MB
Format
Adobe PDF
Checksum (MD5)
1f2f5085d56aca8fc1c90325ae17a8bf