Title
Experiences and Challenges of Working with AI - A Panel Discussion
Panelists
Mancy Liao

Mancy Liao is a Staff Measurement Scientist at Duolingo, where she co-leads the Test Scoring team for the Duolingo English Test. Her work combines psychometrics with AI and machine learning, and her research covers item calibration, computerized adaptive testing, score quality assurance and test security. She works closely with AI researchers and engineers to integrate methodological innovations into large-scale operational language testing, such as automated systems for new item piloting and calibration and for score quality assurance. Profile
Tiago Calico

Tiago Calico is an Analyst in the PISA Research, Development, and Innovation team. His areas of work are using process data for session validation, scoring and latent variable modelling, automated scoring, and data visualization. Tiago has extensive experience writing software for large-data analysis and visualization and has integrated AI into his team-wide workflows. PISA profile reference
Xiaying James Zheng

Xiaying James Zheng is a senior researcher and psychometrician at the American Institutes for Research (AIR). His expertise includes item response theory, latent variable models, process data analysis, missing data, and analysis of complex sample data. Dr. Zheng leads AIR’s technical review portfolio for the National Assessment of Educational Progress (NAEP), overseeing reviews of assessment data, reporting products, and methodological documentation. He has also worked on the U.S. Food and Drug Administration (FDA) contracts. AIR profile
Moderator
Peter M Steiner, University of Maryland
Abstract
Mancy Liao (Duolingo), Tiago Calico (OECD/PISA), and Xiaying James Zheng (AIR), three alumni of the University of Maryland’s Quantitative Methodology: Measurement and Statistics (QMMS) program, will form a panel to discuss their experiences and challenges of working with artificial intelligence (AI) in their professional roles as statisticians, psychometricians, and quantitative researchers. As AI is increasingly integrated into research and data analysis, it creates new opportunities for automation, efficiency, and scientific discovery, but also raises important questions about the validity, interpretability, and trustworthiness of AI-generated results. Drawing on their professional experiences, the panelists will discuss how they use AI in their work, the opportunities and challenges they encounter, and how AI is changing the skills and expertise required of quantitative methodologists. The panel will be moderated by Peter M Steiner, University of Maryland.