Towards principled learner selection
Publication Date
July 15, 2013
Creator
Abstract
Long learner evaluation times are no longer exceptional and often there is insufficient time to exhaustively test all candidate options. When deciding which learners to use, practitioners must rely on ad hoc testing and luck to identify the most accurate one. Given the importance of classification in decision making, this is unsatisfactory. Progress towards a principled approach requires accurate predictions of learner accuracy and evaluation time and this study examines the potential of traditional meta-learning approaches, with their emphasis on indirect explanatory variables, to deliver the required solutions.
Here, 57 different indirect dataset characteristics, including those related to geometrical complexity, are used as explanatory variables, alongside sample-estimates, in building regression models of accuracy and time.
The evidence presented firmly suggests that these indirect variables lack both the required predictive power and the time efficiency required for the development of practically useful models, and points instead towards basing the prediction of learner accuracy solely on sample-based models. The attempt at modelling learner evaluation time reveals some of the difficulties that this tough challenge presents.
Item Type
ethesis
Thesis Type
MRes
Supervisors
Keywords
Subjects (LC)
Associated Schools / Departments
School of Computer Science (UK)
eprints ID
13299
UoN Repository URI
Except where otherwise noted, this item's license is described as
File(s)![Thumbnail Image]()
Name
MRES_ARATTRAY_4145685_FINAL_CORRECTED.pdf
Type
Full-text
Size
455.21 KB
Format
Adobe PDF
Checksum ()