Automated feature engineering, AutoML, and decision-focused learning for improved energy consumption forecasting

Alkhulaifi, Nasser

Automated feature engineering, AutoML, and decision-focused learning for improved energy consumption forecasting

Publication Date

July 29, 2026

Creator

Alkhulaifi, Nasser

University of Nottingham, UK

Abstract

The rising cost and demand for energy, coupled with the need to meet environmental sustainability goals, create pressing challenges for energy management. Energy Consumption Forecasting (ECF) supports informed planning by predicting future consumption patterns, yet Machine Learning (ML) models for ECF remain highly dependent on domain expertise. This dependence is driven largely by manual, expert-driven Feature Engineering (FE), since raw energy data often require preprocessing and transformation before ML algorithms can learn effectively. Moreover, real-world datasets are often small due to data collection limitations, privacy issues, or resource constraints; in such cases, FE can partially compensate by extracting and selecting informative features to maximise the utility of available data and improve predictive performance. While state-of-the-art AutoML frameworks streamline model selection and hyperparameter tuning, they typically assume that data preparation and FE have already been completed. Existing automated FE (AFE) methods are largely domain-agnostic and often fail to capture energy-specific temporal patterns and exogenous effects. In addition, because ECF forecasts usually drive downstream operational decisions, optimising prediction accuracy alone can allow residual errors to propagate into suboptimal actions; Decision-Focused Learning (DFL) methods aim to address this by integrating prediction with the downstream optimisation objective, yet have seen limited real-world evaluation in automated ECF settings. This thesis, therefore, addresses these challenges through three key contributions. First, it establishes and evaluates a comprehensive FE pipeline for ECF, investigates domain-specific features, and provides the empirical foundation for subsequent FE automation efforts. Second, it introduces AutoEnergy, a domain-tailored AFE algorithm that generates interpretable features from timestamps and lagged consumption via rule-based transformations, and integrates with AutoML to enable end-to-end automated ECF modelling. Across eighteen diverse real-world energy datasets spanning residential, commercial, industrial, renewable, and grid power domains, AutoEnergy reduces forecasting error by 19.52%–84.72% relative to baseline AutoML and established AFE methods, while running 1.31–4.41 times faster, with performance gains varying by dataset. Third, it leverages AutoEnergy within a DFL framework for a Battery Energy Storage System (BESS) problem, jointly forecasting electricity prices and demand while optimising a charging and discharging strategy; on a real-world UK property dataset, this AFE–DFL approach reduces operating costs by 22.9%–56.5% compared with the same DFL models without AFE. While the empirical results in this thesis are dataset-dependent, they are drawn from multiple real-world datasets spanning diverse energy settings. This supports two general conclusions: (i) integrating AutoEnergy with AutoML can enable automated ECF modelling that reduces reliance on manual FE while improving forecasting accuracy; and (ii) DFL enhanced with AFE can translate predictive improvements into measurable operational benefits for ECF applications. These contributions have broader implications for energy management systems in settings with limited domain expertise and small datasets, and demonstrate potential to support the transition to automated, AI-based energy systems.

Item Type

ethesis

Thesis Type

PhD

Supervisors

Triguero, Isaac

University of Nottingham

Watson, Nicholas

University of Leeds

Bowler, Alexander

University of Leeds

Pekaslan, Direnc