Search Results
Working Paper
Total Recall? Evaluating the Macroeconomic Knowledge of Large Language Models
We evaluate the ability of large language models (LLMs) to estimate historical macroeconomic variables and data release dates. We find that LLMs have precise knowledge of some recent statistics, but performance degrades as we go farther back in history. We highlight two particularly important kinds of recall errors: mixing together first print data with subsequent revisions (i.e., smoothing across vintages) and mixing data for past and future reference periods (i.e., smoothing within vintages). We also find that LLMs can often recall individual data release dates accurately, but aggregating ...
Working Paper
The perils of working with Big Data and a SMALL framework you can use to avoid them
The use of “Big Data” to explain fluctuations in the broader economy or guide the business decisions of a firm is now so commonplace that in some instances it has even begun to rival more traditional government statistics and business analytics. Big data sources can very often provide advantages when compared to these more traditional data sources, but with these advantages also comes the potential for pitfalls. We lay out a framework called SMALL that we have developed in order to help interested parties as they navigate the big data minefield. Based on a set of five questions, the SMALL ...
Working Paper
Speaking for Herself: Changing Gender Roles in Survey Response
Among married and cohabiting couples, the percentage of female respondents has increased substantially in the PSID (Panel Study of Income Dynamics) from 9% in 1968 to 60% in 2015. This shift in gender composition has taken place despite a formal policy that historically designated male heads of household as respondents. We use this shift as a case study to explore which characteristics are associated with women responding to the PSID and how different respondent gender compositions may affect data quality. First, we find that women are increasingly less likely to respond as their husband?s ...
Report
Fed Transparency and Policy Expectation Errors: A Text Analysis Approach
This paper seeks to estimate the extent to which market-implied policy expectations could be improved with further information disclosure from the FOMC. Using text analysis methods based on large language models, we show that if FOMC meeting materials with five-year lagged release dates—like meeting transcripts and Tealbooks—were accessible to the public in real time, market policy expectations could substantially improve forecasting accuracy. Most of this improvement occurs during easing cycles. For instance, at the six-month forecasting horizon, the market could have predicted as much ...
Speech
Remarks at the fifth Data Management Strategies and Technologies Workshop
Remarks at the Fifth Data Management Strategies and Technologies Workshop, Federal Reserve Bank of New York, New York City
Working Paper
Shedding Light on Survey Accuracy—A Comparison between SHED and Census Bureau Survey Results
The annual Survey of Household Economics and Decisionmaking (SHED) receives substantial research attention for topics related to household finances and economic well-being. To assess the reliability of data from the SHED, we compare aggregate statistics from the SHED with prominent, nationally representative surveys that use different survey designs, sample methodologies, and interview modes. Specifically, we compare recent statistics from the SHED with similar questions in U.S. Census Bureau surveys, including the Current Population Survey (CPS) and the American Community Survey (ACS). ...
Working Paper
Estimating U.S. Cross-Border Securities Positions: New Data and New Methods
The role of capital flows in the buildup to the global financial crisis and the potential vulnerabilities posed by capital flows to emerging market economies highlight the importance of reliable and timely measures of cross-border investment activity to better monitor developments as they unfold. We present new monthly estimates of U.S. cross-border securities investment, combining information from detailed annual Treasury International Capital (TIC) surveys with new information from the TIC form SLT. We also show how changes in the new monthly data can be decomposed into flows, estimated ...
Report
Measuring the US Employment Situation Using Online Panels: The Yale Labor Survey
This report presents the results of a rapid, low-cost survey that collects labor market data for individuals in the United States. The Yale Labor Survey (YLS) used an online panel from YouGov to replicate statistics from the Current Population Survey (CPS), the government’s source of household labor market statistics. The YLS’s advantages include its timeliness, low cost, and ability to develop new questions quickly to study labor market patterns during the coronavirus (COVID-19) pandemic. Although YLS estimates of unemployment and participation rates mirrored the broad trends in CPS ...
Working Paper
The perils of working with Big Data and a SMALL framework you can use to avoid them
The use of “Big Data” to explain fluctuations in the broader economy or guide the business decisions of a firm is now so commonplace that in some instances it has even begun to rival more traditional government statistics and business analytics. Big data sources can very often provide advantages when compared to these more traditional data sources, but with these advantages also comes the potential for pitfalls. We lay out a framework called SMALL that we have developed in order to help interested parties as they navigate the big data minefield. Based on a set of five questions, the SMALL ...