Search Results
Working Paper
Evaluating Local Language Models: An Application to Bank Earnings Calls
This study evaluates the performance of local large language models (LLMs) in interpreting financial texts, compared with closed-source, cloud-based models. We first introduce new benchmarking tasks for assessing LLM performance in analyzing financial and economic texts and explore the refinements needed to improve its performance. Our benchmarking results suggest local LLMs are a viable tool for general natural language processing analysis of these texts. We then leverage local LLMs to analyze the tone and substance of bank earnings calls in the post-pandemic era, including calls conducted ...
Working Paper
What Do LLMs Want?
Large language models (LLMs) are now used for economic reasoning, but their implicit "preferences” are poorly understood. We study LLM preferences as revealed by their choices in simple allocation games and a job-search setting. Most models favor equal splits in dictator-style allocation games, consistent with inequality aversion. Structural estimates recover Fehr–Schmidt parameters that indicate inequality aversion is stronger than in similar experiments with human participants. However, we find these preferences are malleable: reframing (e.g., masking social context) and learned ...
Working Paper
What Do LLMs Want?
Large language models (LLMs) are now used for economic reasoning, but their implicit "preferences" are poorly understood. We study these preferences by analyzing revealed choices in canonical allocation games and a sequential job-search environment. In dictator-style allocation games, most models favor equal splits, consistent with inequality aversion. Structural estimation of Fehr-Schmidt parameters suggests this aversion exceeds levels typically observed in human experiments. However, LLM preferences prove malleable. Interventions such as prompt framing (e.g., masking social context) and ...