Working Paper

Missing Data Substitution for Enhanced Robust Filtering and Forecasting in Linear State-Space Models

Abstract: Replacing faulty measurements with missing values can suppress outlier-induced distortions in state-space inference. We therefore put forward two complementary methods for enhanced outlier-robust filtering and forecasting: supervised missing data substitution (MD) upon exceeding a Huber threshold, and unsupervised missing data substitution via exogenous randomization (RMDX).Our supervised method, MD, is designed to improve performance of existing Huber-based linear filters known to lose optimality when outliers of the same sign are clustered in time rather than arriving independently. The unsupervised method, RMDX, further aims to suppress smaller outliers whose size may fall below the Huber detection threshold. To this end, RMDX averages filtered or forecasted targets based on measurement series with randomly induced subsets of missing data at an exogenously set randomization rate. This gives rise to regularization and bias-variance trade-off as a function of the missing data randomization rate, which can be set optimally using standard cross-validation techniques.We validate through Monte Carlo simulations that both methods for missing data substitution can significantly improve robust filtering, especially when combined together. As further empirical validation, we document consistently attractive performance in linear models for forecasting inflation trends prone to clustering of measurement outliers.

Keywords: Kalman filter; Outliers; Huberization; Missing data; Randomization;

JEL Classification: C15; C22; C53; E37;

https://doi.org/10.17016/FEDS.2025.001

Bibliographic Information

Provider: Board of Governors of the Federal Reserve System (U.S.)

Part of Series: Finance and Economics Discussion Series

Publication Date: 2025-01-03

Number: 2025-001