Search Results
Working Paper
Explaining Machine Learning by Bootstrapping Partial Marginal Effects and Shapley Values
Machine learning and artificial intelligence are often described as “black boxes.” Traditional linear regression is interpreted through its marginal relationships as captured by regression coefficients. We show that the same marginal relationship can be described rigorously for any machine learning model by calculating the slope of the partial dependence functions, which we call the partial marginal effect (PME). We prove that the PME of OLS is analytically equivalent to the OLS regression coefficient. Boot- strapping provides standard errors and confidence intervals around the point ...
Working Paper
Integrating Prediction and Attribution to Classify News
Recent modeling developments have created tradeoffs between attribution-based models, models that rely on causal relationships, and “pure prediction models†such as neural networks. While forecasters have historically favored one technology or the other based on comfort or loyalty to a particular paradigm, in domains with many observations and predictors such as textual analysis, the tradeoffs between attribution and prediction have become too large to ignore. We document these tradeoffs in the context of relabeling 27 million Thomson Reuters news articles published between 1996 ...
Working Paper
Macroeconomic Indicator Forecasting with Deep Neural Networks
Economic policymaking relies upon accurate forecasts of economic conditions. Current methods for unconditional forecasting are dominated by inherently linear models {{p}} that exhibit model dependence and have high data demands. {{p}} We explore deep neural networks as an {{p}} opportunity to improve upon forecast accuracy with limited data and while remaining agnostic as to {{p}} functional form. We focus on predicting civilian unemployment using models based on four different neural network architectures. Each of these models outperforms bench- mark models at short time horizons. One model, ...
Working Paper
Generative AI at the Crossroads: Light Bulb, Dynamo, or Microscope?
With the advent of generative AI (genAI), the potential scope of artificial intelligence has increased dramatically, but the future effect of genAI on productivity remains uncertain. The effect of the technology on the innovation process is a crucial open question. Some inventions, such as the light bulb, temporarily raise productivity growth as adoption spreads, but the effect fades when the market is saturated; that is, the level of output per hour is permanently higher but the growth rate is not. In contrast, two types of technologies stand out as having longer-lived effects on ...
Working Paper
Artificial Intelligence and Inflation Forecasts
We explore the ability of Large Language Models (LLMs) to produce in-sample conditional inflation forecasts during the 2019-2023 period. We use a leading LLM (Google AI's PaLM) to produce distributions of conditional forecasts at different horizons and compare these forecasts to those of a leading source, the Survey of Professional Forecasters (SPF). We find that LLM forecasts generate lower mean-squared errors overall in most years, and at almost all horizons. LLM forecasts exhibit slower reversion to the 2% inflation anchor.
Working Paper
Artificial Intelligence and Inflation Forecasts
We explore the ability of Large Language Models (LLMs) to produce conditional inflation forecasts during the 2019-2023 period. We use a leading LLM (Google AI's PaLM) to produce distributions of conditional forecasts at different horizons and compare these forecasts to those of a leading source, the Survey of Professional Forecasters (SPF). We find that LLM forecasts generate lower mean-squared errors overall in most years, and at almost all horizons. LLM forecasts exhibit slower reversion to the 2% inflation anchor. We argue that this method of generating forecasts is inexpensive and can be ...
Working Paper
Spatial Dependence and Data-Driven Networks of International Banks
This paper computes data-driven correlation networks based on the stock returns of international banks and conducts a comprehensive analysis of their topological properties. We first apply spatial-dependence methods to filter the effects of strong common factors and a thresholding procedure to select the significant bilateral correlations. The analysis of topological characteristics of the resulting correlation networks shows many common features that have been documented in the recent literature but were obtained with private information on banks? exposures. Our analysis validates these ...
Working Paper
The Anatomy of Out-of-Sample Forecasting Accuracy
We develop metrics based on Shapley values for interpreting time-series forecasting models, including“black-box” models from machine learning. Our metrics are model agnostic, so that they are applicable to any model (linear or nonlinear, parametric or nonparametric). Two of the metrics, iShapley-VI and oShapley-VI, measure the importance of individual predictors in fitted models for explaining the in-sample and out-of-sample predicted target values, respectively. The third metric is the performance-based Shapley value (PBSV), our main methodological contribution. PBSV measures the ...