Skip to Main Content (Press Enter)

Logo UNIOR
  • ×
  • Home
  • Corsi
  • Insegnamenti
  • Persone
  • Strutture
  • Attrezzature

UNIFIND
Logo UNIOR

|

UNIFIND

unior.it
  • ×
  • Home
  • Corsi
  • Insegnamenti
  • Persone
  • Strutture
  • Attrezzature

Riddle Me This: Evaluating Large Language Models in Solving Word-Based Games

Contributo in Atti di convegno
Data di Pubblicazione:
2024
Abstract:
In this contribution, we examine the proficiency of Large Language Models (LLMs) in solving the linguistic game "La Ghigliottina," the final game of the popular Italian TV quiz show “L’Eredità”. This game is particularly challenging as it requires LLMs to engage in semantic inference reasoning for identifying the solutions of the game. Our experiment draws inspiration from Ghigliottin-AI, a task of EVALITA 2020, an evaluation campaign focusing on Natural Language Processing (NLP) and speech tools designed for the Italian language. To benchmark our experiment, we use the results of the most successful artificial player in this task, namely Il Mago della Ghigliottina. The paper describes the experimental setting and the results which show that LLMs perform poorly.
Tipologia CRIS:
4.1 Contributo in Atti di convegno
Elenco autori:
Manna, Raffaele; di Buono, Maria Pia; Monti, Johanna
Autori di Ateneo:
MANNA RAFFAELE
MONTI JOHANNA
di Buono Maria Pia
Link alla scheda completa:
https://unora.unior.it/handle/11574/227701
Titolo del libro:
Proceedings of Games and NLP Workshop - LREC-COLING 2024
  • Utilizzo dei cookie

Realizzato con VIVO | Designed by Cineca | 26.11.1.0