University of Hertfordshire

By the same authors

The misuse of the NASA metrics data program data sets for automated software defect prediction

Research output: Chapter in Book/Report/Conference proceedingConference contribution


  • 905745

    Accepted author manuscript, 208 KB, PDF document

View graph of relations
Original languageEnglish
Title of host publicationProcs 15th Annual Conference on Evaluation & Assessment in Software Engineering (EASE 2011)
ISBN (Print)978-1-84919-509-6
Publication statusPublished - 2011
EventProceedings of the 15th International Conference on Evaluation and Assessment in Software Engineering - Durham, United Kingdom
Duration: 11 Apr 201112 Apr 2011


ConferenceProceedings of the 15th International Conference on Evaluation and Assessment in Software Engineering
CountryUnited Kingdom


Background: The NASA Metrics Data Program data sets have been heavily used in software defect prediction experiments.
Aim: To demonstrate and explain why these data sets require significant pre-processing in order to be suitable for defect prediction.
Method: A meticulously documented data cleansing process involving all 13 of the original NASA data sets.
Results: Post our novel data cleansing process; each of the data sets had between 6 to 90 percent less of their original number of recorded values.
One: Researchers need to analyse the data that forms the basis of their findings in the context of how it will be used.
Two: Defect prediction data sets could benefit from lower level code metrics in addition to those more commonly used, as these will help to distinguish modules, reducing the likelihood of repeated data points.
Three: The bulk of defect prediction experiments based on the NASA Metrics Data Program data sets may have led to erroneous findings. This is mainly due to repeated data points potentially causing substantial amounts of training and testing data to be identical.

ID: 1406711