Search Results
Working Paper
Validating Large Language Model Annotations
This paper proposes a validation framework for LLM-generated measurements when reliable benchmarks are unavailable. Validity is established by testing whether an LLM can reconstruct passages from annotated labels while maintaining semantic consistency with the original text. The framework avoids circular reasoning by establishing testable prerequisite properties that must be met for a validation to be considered successful. Application to news article data demonstrates that the framework serves as a practical alternative to human benchmarking, which offers advantages in objectivity, ...
Working Paper
Artificial Intelligence Methods for Evaluating Global Trade Flows
International trade policies remain in the spotlight given the recent rethink on the benefits of globalization by major economies. Since trade critically affects employment, production, prices and wages, understanding and predicting future patterns of trade is a high-priority for decision making within and across countries. While traditional economic models aim to be reliable predictors, we consider the possibility that Artificial Intelligence (AI) techniques allow for better predictions and associations to inform policy decisions. Moreover, we outline contextual AI methods to decipher trade ...
Working Paper
The perils of working with Big Data and a SMALL framework you can use to avoid them
The use of “Big Data” to explain fluctuations in the broader economy or guide the business decisions of a firm is now so commonplace that in some instances it has even begun to rival more traditional government statistics and business analytics. Big data sources can very often provide advantages when compared to these more traditional data sources, but with these advantages also comes the potential for pitfalls. We lay out a framework called SMALL that we have developed in order to help interested parties as they navigate the big data minefield. Based on a set of five questions, the SMALL ...
Report
Fed Transparency and Policy Expectation Errors: A Text Analysis Approach
This paper seeks to estimate the extent to which market-implied policy expectations could be improved with further information disclosure from the FOMC. Using text analysis methods based on large language models, we show that if FOMC meeting materials with five-year lagged release dates—like meeting transcripts and Tealbooks—were accessible to the public in real time, market policy expectations could substantially improve forecasting accuracy. Most of this improvement occurs during easing cycles. For instance, at the six-month forecasting horizon, the market could have predicted as much ...
Working Paper
The perils of working with Big Data and a SMALL framework you can use to avoid them
The use of “Big Data” to explain fluctuations in the broader economy or guide the business decisions of a firm is now so commonplace that in some instances it has even begun to rival more traditional government statistics and business analytics. Big data sources can very often provide advantages when compared to these more traditional data sources, but with these advantages also comes the potential for pitfalls. We lay out a framework called SMALL that we have developed in order to help interested parties as they navigate the big data minefield. Based on a set of five questions, the SMALL ...
Speech
Remarks at the Fifth Data Management Strategies and Technologies Workshop
Remarks at the Fifth Data Management Strategies and Technologies Workshop, Federal Reserve Bank of New York, New York City
Working Paper
Database of global economic indicators (DGEI): a methodological note
The Database of Global Economic Indicators (DGEI) from the Federal Reserve Bank of Dallas is aimed at standardizing and disseminating world economic indicators for policy analysis and scholarly work on the role of globalization. The purpose of DGEI is to offer a broad perspective on how economic developments around the world influence the U.S. economy with a wide selection of indicators. DGEI is automated within an Excel-VBA and E-views framework for the processing and aggregation of multiple country time series. It includes a core sample of 40 countries with available indicators and broad ...
Working Paper
Measuring Movement and Social Contact with Smartphone Data: A Real-Time Application to COVID-19
Tracking human activity in real time and at fine spatial scale is particularly valuable during episodes such as the COVID-19 pandemic. In this paper, we discuss the suitability of smartphone data for quantifying movement and social contact. We show that these data cover broad sections of the US population and exhibit movement patterns similar to conventional survey data. We develop and make publicly available a location exposure index that summarizes county-to-county movements and a device exposure index that quantifies social contact within venues. We use these indices to document how ...
Working Paper
Estimating U.S. Cross-Border Securities Positions: New Data and New Methods
The role of capital flows in the buildup to the global financial crisis and the potential vulnerabilities posed by capital flows to emerging market economies highlight the importance of reliable and timely measures of cross-border investment activity to better monitor developments as they unfold. We present new monthly estimates of U.S. cross-border securities investment, combining information from detailed annual Treasury International Capital (TIC) surveys with new information from the TIC form SLT. We also show how changes in the new monthly data can be decomposed into flows, estimated ...
Working Paper
Harmonized Population and Labor Force Statistics
The official labor force statistics often exhibit discontinuities in January, when updated population estimates are incorporated into the Current Population Survey (CPS) for the current year but are not revised backward through history. We construct harmonized population estimates spanning five decades and produce new weights for the CPS microdata that are benchmarked to these estimates. Using these weights, we estimate harmonized labor force statistics that reflect the latest available information about the population and its characteristics. The harmonized labor force series are free from ...