Working Paper

Applications of Markov Chain Approximation Methods to Optimal Control Problems in Economics


Abstract: In this paper we explore some of the benefits of using the finite-state Markov chain approximation (MCA) method of Kushner and Dupuis (2001) to solve continuous-time optimal control problems. We first show that the implicit finite-difference scheme of Achdou et al. (2017) amounts to a limiting form of the MCA method for a certain choice of approximating chains and policy function iteration for the resulting system of equations. We then illustrate the benefits of departing from policy function iteration by showing that using variations of modified policy function iteration to solve income fluctuation problems in two and three dimensions can lead to an increase in the speed of convergence of more than an order of magnitude. We then show that the MCA method is also well-suited to solving portfolio problems with highly correlated state variables, a setting that commonly occurs within general equilibrium models with financial frictions and for which it is difficult to construct monotone (and hence convergent) finite-difference schemes.

Keywords: Dynamic programming; financial frictions;

JEL Classification: C63; E00; G11;

https://doi.org/10.26509/frbc-wp-202104

Access Documents

File(s): File format is text/html https://doi.org/10.26509/frbc-wp-202104
Description: Full Text

Authors

Bibliographic Information

Provider: Federal Reserve Bank of Cleveland

Part of Series: Working Papers

Publication Date: 2021-02-10

Number: 202104