Search Results
Working Paper
Explaining Machine Learning by Bootstrapping Partial Marginal Effects and Shapley Values
Machine learning and artificial intelligence are often described as “black boxes.” Traditional linear regression is interpreted through its marginal relationships as captured by regression coefficients. We show that the same marginal relationship can be described rigorously for any machine learning model by calculating the slope of the partial dependence functions, which we call the partial marginal effect (PME). We prove that the PME of OLS is analytically equivalent to the OLS regression coefficient. Boot- strapping provides standard errors and confidence intervals around the point ...
Working Paper
Macroeconomic Indicator Forecasting with Deep Neural Networks
Economic policymaking relies upon accurate forecasts of economic conditions. Current methods for unconditional forecasting are dominated by inherently linear models {{p}} that exhibit model dependence and have high data demands. {{p}} We explore deep neural networks as an {{p}} opportunity to improve upon forecast accuracy with limited data and while remaining agnostic as to {{p}} functional form. We focus on predicting civilian unemployment using models based on four different neural network architectures. Each of these models outperforms bench- mark models at short time horizons. One model, ...
Journal Article
Testing Hybrid Forecasts for Imports and Exports
The quality of economic forecasts tends to deteriorate during times of stress such as the COVID-19 pandemic, raising questions about how to improve forecasts during exceptional times. One method of forecasting that has received less attention is refining model-based forecasts with judgmental adjustment, or hybrid forecasting. Judgmental adjustment is the process of incorporating information from outside a model into a forecast or adjusting a forecast subjectively. Hybrid forecasts could be particularly useful during extraordinary times such as the COVID-19 pandemic, as models that do not ...
Working Paper
Assessing Macroeconomic Tail Risks in a Data-Rich Environment
We use a large set of economic and financial indicators to assess tail risks of the three macroeconomic variables: real GDP, unemployment, and inflation. When applied to U.S. data, we find evidence that a dense model using principal components (PC) as predictors might be misspecified by imposing the “common slope” assumption on the set of predictors across multiple quantiles. The common slope assumption ignores the heterogeneous informativeness of individual predictors on different quantiles. However, the parsimony of the PC-based approach improves the accuracy of out-of-sample forecasts ...
Working Paper
Explaining Machine Learning by Bootstrapping Partial Marginal Effects and Shapley Values
Machine learning and artificial intelligence are often described as “black boxes.” Traditional linear regression is interpreted through its marginal relationships as captured by regression coefficients. We show that the same marginal relationship can be described rigorously for any machine learning model by calculating the slope of the partial dependence functions, which we call the partial marginal effect (PME). We prove that the PME of OLS is analytically equivalent to the OLS regression coefficient. Bootstrapping provides standard errors and confidence intervals around the point ...
Working Paper
Evaluating Local Language Models: An Application to Bank Earnings Calls
This study evaluates the performance of local large language models (LLMs) in interpreting financial texts, compared with closed-source, cloud-based models. We first introduce new benchmarking tasks for assessing LLM performance in analyzing financial and economic texts and explore the refinements needed to improve its performance. Our benchmarking results suggest local LLMs are a viable tool for general natural language processing analysis of these texts. We then leverage local LLMs to analyze the tone and substance of bank earnings calls in the post-pandemic era, including calls conducted ...
Working Paper
What Do LLMs Want?
Large language models (LLMs) are now used for economic reasoning, but their implicit "preferences” are poorly understood. We study LLM preferences as revealed by their choices in simple allocation games and a job-search setting. Most models favor equal splits in dictator-style allocation games, consistent with inequality aversion. Structural estimates recover Fehr–Schmidt parameters that indicate inequality aversion is stronger than in similar experiments with human participants. However, we find these preferences are malleable: reframing (e.g., masking social context) and learned ...
Working Paper
Explaining Machine Learning by Bootstrapping Partial Dependence Functions and Shapley Values
Machine learning and artificial intelligence methods are often referred to as “black boxes” when compared with traditional regression-based approaches. However, both traditional and machine learning methods are concerned with modeling the joint distribution between endogenous (target) and exogenous (input) variables. Where linear models describe the fitted relationship between the target and input variables via the slope of that relationship (coefficient estimates), the same fitted relationship can be described rigorously for any machine learning model by first-differencing the partial ...
Journal Article
Assessing the Risk of Extreme Unemployment Outcomes
Although the unemployment rate is at a historically low level, many policymakers are nevertheless watching projections for the future unemployment rate closely to evaluate the risk of extreme outcomes. We assess the probabilities of extreme outcomes in the near and medium term and find that the risk of unexpectedly high unemployment three years in the future has declined from its Great Recession peak and remained low over the past three years.