Search Results
Working Paper
Financial Stability Implications of Generative AI: Taming the Animal Spirits
This paper investigates the impact of the adoption of generative AI on financial stability. We conduct laboratory-style experiments using large language models to replicate classic studies on herd behavior in investment decisions. Our results show that AI agents make more rational decisions than humans, relying predominantly on private information over market trends. Increased reliance on AI-powered investment advice could therefore potentially lead to fewer asset price bubbles arising from animal spirits that trade by following the herd. However, exploring variations in the experimental ...
Working Paper
Evaluating Local Language Models: An Application to Bank Earnings Calls
This study evaluates the performance of local large language models (LLMs) in interpreting financial texts, compared with closed-source, cloud-based models. We first introduce new benchmarking tasks for assessing LLM performance in analyzing financial and economic texts and explore the refinements needed to improve its performance. Our benchmarking results suggest local LLMs are a viable tool for general natural language processing analysis of these texts. We then leverage local LLMs to analyze the tone and substance of bank earnings calls in the post-pandemic era, including calls conducted ...
Working Paper
Designing Market Shock Scenarios
We propose an approach for generating financial market scenarios for stress testing financial firms' market risk exposures. This approach can be used by industry practitioners and regulators for their stress scenario design. Our approach attempts to maximize risk capture with a relatively small number of scenarios. A single scenario could miss potential vulnerabilities, while stress tests using a large number of scenarios could be operationally costly. The approach has two components. First, we model relationships among market risk factors to set scenario shock magnitudes consistently across ...
Working Paper
Validating Large Language Model Annotations
This paper proposes a validation framework for LLM-generated measurements when reliable benchmarks are unavailable. Validity is established by testing whether an LLM can reconstruct passages from annotated labels while maintaining semantic consistency with the original text. The framework avoids circular reasoning by establishing testable prerequisite properties that must be met for a validation to be considered successful. Application to news article data demonstrates that the framework serves as a practical alternative to human benchmarking, which offers advantages in objectivity, ...
Working Paper
Validating Large Language Model Annotations
This paper proposes a validation framework for LLM-generated measurements when reliable benchmarks are unavailable. Validity is established by testing whether an LLM can reconstruct passages from annotated labels while maintaining semantic consistency with the original text. The framework avoids circular reasoning by establishing testable prerequisite properties that must be met for a validation to be considered successful. Application to news article data demonstrates that the framework serves as a practical alternative to human benchmarking, which offers advantages in objectivity, ...