Search Results
Working Paper
Total Recall? Evaluating the Macroeconomic Knowledge of Large Language Models
We evaluate the ability of large language models (LLMs) to estimate historical macroeconomic variables and data release dates. We find that LLMs have precise knowledge of some recent statistics, but performance degrades as we go farther back in history. We highlight two particularly important kinds of recall errors: mixing together first print data with subsequent revisions (i.e., smoothing across vintages) and mixing data for past and future reference periods (i.e., smoothing within vintages). We also find that LLMs can often recall individual data release dates accurately, but aggregating ...
Working Paper
Manufacturing Sentiment: Forecasting Industrial Production with Text Analysis
This paper examines the link between industrial production and the sentiment expressed in natural language survey responses from U.S. manufacturing firms. We compare several natural language processing (NLP) techniques for classifying sentiment, ranging from dictionary-based methods to modern deep learning methods. Using a manually labeled sample as ground truth, we find that deep learning models partially trained on a human-labeled sample of our data outperform other methods for classifying the sentiment of survey responses. Further, we capitalize on the panel nature of the data to train ...
Working Paper
Generative AI at the Crossroads: Light Bulb, Dynamo, or Microscope?
With the advent of generative AI (genAI), the potential scope of artificial intelligence has increased dramatically, but the future effect of genAI on productivity remains uncertain. The effect of the technology on the innovation process is a crucial open question. Some inventions, such as the light bulb, temporarily raise productivity growth as adoption spreads, but the effect fades when the market is saturated; that is, the level of output per hour is permanently higher but the growth rate is not. In contrast, two types of technologies stand out as having longer-lived effects on ...
Discussion Paper
Measuring AI Uptake in the Workplace
Artificial Intelligence (AI) may be poised to raise productivity across various domains, including writing (Noy and Zhang 2023), programming (Peng et al. 2023), and research and development (Toner-Rodgers 2024; Korinek 2023). However, understanding the extent to which AI—and generative AI in particular—has been adopted as part of the production process remains an open question. This note reviews the extant surveys on AI adoption at both the employee and firm levels. Surveys of firms show a wide spread of adoption rates, ranging from 5 percent to about 40 percent. Surveys of workers show ...
Working Paper
Tracking Real Time Layoffs with SEC Filings: A Preliminary Investigation
We explore a new source of data on layoffs: timely 8-K filings with the Securities and and Exchange Commission. We develop measures of both the number of reported layoff events and the number of affected workers. These series are highly correlated with the business cycle and other layoff indicators. Linking firm-level reported layoff events with WARN notices suggests that 8-K filings are sometimes available before WARN notices, and preliminary regression results suggest our layoff series are useful for forecasting. We also document the industry composition of the data and specific areas ...