Имамо задовољство да Вас позовемо на предавање у оквиру Семинара Друштва ЈеРТех. Предавање на тему Validation of LLM-Agent Social Simulations: Toxicity, Semantic Similarity, Topic Dynamics, and Convergence одржаће Александар Томашевић са Института за физику 13. новембра 2025. године од 18 часова на Рударско-геолошком факултету (Ђушина 7).
Следи сажетак излагања:
Online social platforms shape public discourse, yet studying their dynamics through traditional methods faces ethical, access, and reproducibility constraints in the post-API era. Agent-based simulations offer a promising alternative, enabling controlled experiments on synthetic populations. While traditional agent-based models rely on rule-based agents optimizing predefined objectives, large language models (LLMs) can simulate human-like behavior through natural language interactions. However, the question of operational validity remains: do LLM-agent simulations reproduce key patterns observed in real-world data?
We investigate this question through a baseline approach based a 30-day simulation of a Reddit-like technology forum using YSocial. YSocial deploys LLM agents (Dolphin 3.0, Llama 3.1‑8B) based on persona-prompts, interacting in a structured social media environment. The simulation is calibrated to a real-world dataset from the Voat platform, focusing on technology-related discussions. The study evaluates operational validity through four key dimensions: toxicity, semantic similarity, discussion topics, and linguistic convergence.
Results suggest interpretable checkpoints for operational validity. The simulation reproduces recognizable topical structure and coherent semantic neighborhoods, while toxicity is elevated with a heavier upper tail than the reference data. Language convergence appears largely local, agents align in immediate exchanges but drift beyond the visible thread lengths, consistent with the absence of agent memory. Overall, even a small, memoryless setup achieves reasonable validity while clarifying current limits and directions for improvement.
