Welcome to

Lancashire Online Knowledge

Image Credit Header image: Artwork by Professor Lubaina Himid, CBE. Photo: @Denise Swanson


Enhancing Digital Heritage Experiences: Evaluating Fine-Tuned LLM Integration within a Cyber-Physical-Social Virtual Museum System

Nisiotis, Louis orcid iconORCID: 0000-0002-8018-1352, Markov, Nikita, Nikolaou, Charalampos, Hadjiliasi, Aimilios and Raspopoulos, Marios orcid iconORCID: 0000-0003-1513-6018 (2026) Enhancing Digital Heritage Experiences: Evaluating Fine-Tuned LLM Integration within a Cyber-Physical-Social Virtual Museum System. In: 2025 IEEE International Symposium on Emerging Metaverse (ISEMV). Institute of Electrical and Electronics Engineers (IEEE). ISBN 979-8-3315-4891-9

[thumbnail of _CRM__ISMEV_2025__LLM_Evaluation_ (1).pdf] PDF - Accepted Version
Restricted to Repository staff only

1MB

Official URL: https://doi.org/10.1109/ISEMV67326.2025.00016

Abstract

The Metaverse is rapidly advancing, creating new ways to connect real and digital spaces. In Digital Cultural Heritage, technologies such as Artificial Intelligence, eXtended Reality, and Digital Twins are transforming how cultural knowledge is preserved and experienced. Cyber-Physical-Social Systems (CPSS) provide a foundation for such applications, yet the role of Large Language Models (LLMs) in these systems remains underexplored. This paper presents the architecture and development of a CPSS-based virtual museum prototype and reports on a comparative evaluation of LLM integration for contextual information delivery. We fine-tuned Mistral 7B with LoRA on domain-specific data and compared it against the baseline model through expert human evaluation on factual accuracy, relevance, safety, alignment, and formatting. Results show the fine-tuned model outperformed the baseline by ≈+5.5%, particularly in accuracy, contextual relevance, and response formatting. These findings demonstrate the value of tailored LLMs as a core component of CPSS for enhancing digital heritage experiences and create new ways for future exploration of additional models and approaches.


Repository Staff Only: item control page