Applications of Markov Chain Approximation Methods to Optimal Control Problems in Economics

RePEc Short ID: pph158
ORCID iD: 0000-0002-5904-5065

Eslami, Keyvan; Phelan, Tom

doi:10.26509/frbc-wp-202104r

Working Paper Revision

Applications of Markov Chain Approximation Methods to Optimal Control Problems in Economics

Abstract: In this paper we explore some benefits of using the finite-state Markov chain approximation (MCA) method of Kushner and Dupuis (2001) to solve continuous-time optimal control problems in economics. We first show that the implicit finite-difference scheme of Achdou et al. (2022) amounts to a limiting form of the MCA method for a certain choice of approximating chains and policy function iteration for the resulting system of equations. We then illustrate that, relative to the implicit finite-difference approach, using variations of modified policy function iteration to solve income fluctuation problems both with and without discrete choices can lead to an increase in the speed of convergence of more than an order of magnitude. Finally, we provide several consistent chain constructions for stationary portfolio problems with correlated state variables, and illustrate the flexibility of the MCA approach by using it to construct and compare two simple solution methods for a general equilibrium model with financial frictions.

Keywords: Dynamic programming; financial frictions; Numerical methods; Markov chain approximation;

JEL Classification: C63; E00; G11;

https://doi.org/10.26509/frbc-wp-202104r