Research

Think Globally, Group Locally: Evaluating LLMs Using Multi-Lingual Word Grouping Games
César Guerra-Solano, Zhuochun Li, Xiang Lorraine Li
EMNLP Main 2025 | link
We design a flexible, resource-efficient multilingual abstract reasoning benchmark, and find LLM performance deficits related to language, cultural context, and domain.

---

Persona Non Grata: LLM Persona-Driven Generations in MCQA are Unstable in Distinct Dimensions
César Guerra-Solano, Xiang Lorraine Li
Preprint. Under review. | link
We design metrics to evaluate instability in persona-driven MCQA across three distinct dimensions, finding consistent patterns with hyperparameter type, choice, domain, and model performance. Importantly, these metrics can be used to inform future experiment design, even if used on a small batch of pilot data.

---