Working Paper Revision

Applications of Markov Chain Approximation Methods to Optimal Control Problems in Economics

Abstract: In this paper we explore some benefits of using the finite-state Markov chain approximation (MCA) method of Kushner and Dupuis (2001) to solve continuous-time optimal control problems in economics. We first show that the implicit finite-difference scheme of Achdou et al. (2022) amounts to a limiting form of the MCA method for a certain choice of approximating chains and policy function iteration for the resulting system of equations. We then illustrate that, relative to the implicit finite-difference approach, using variations of modified policy function iteration to solve income fluctuation problems both with and without discrete choices can lead to an increase in the speed of convergence of more than an order of magnitude. Finally, we provide several consistent chain constructions for stationary portfolio problems with correlated state variables, and illustrate the flexibility of the MCA approach by using it to construct and compare two simple solution methods for a general equilibrium model with financial frictions.

Keywords: Dynamic programming; financial frictions; Numerical methods; Markov chain approximation;

JEL Classification: C63; E00; G11;

https://doi.org/10.26509/frbc-wp-202104r

Bibliographic Information

Provider: Federal Reserve Bank of Cleveland

Part of Series: Working Papers

Publication Date: 2022-05-17

Number: 21-04R

Note: Replication materials may be found at https://github.com/tphelanECON/EslamiPhelan_MCA

Related Works

• Working Paper Revision (2022-05-17) : You are here.
• Working Paper Original (2021-02-10) : Applications of Markov Chain Approximation Methods to Optimal Control Problems in Economics