Cultural Heritage experiences through socio-personal interactions and storytelling

Description: 

The CHESS System is a research prototype that has been developed in the context of the CHESS. It aims to enrich museum visits through personalized interactive storytelling, by (re-)injecting the sense of discovery and wonder in the visitors' experience. It uses personalized information to create customized stories that guide visitors through a museum and employs mixed reality and pervasive games techniques, ranging from narrations to Augmented Reality (AR) on mobile devices.

CHESS targets two “types” of users; visitors, who “consume” CHESS stories through their devices, and story authors, who design the experiences. Aspiring to replace the traditional set of exhibit-centric descriptions by story-centric cohesive narrations with carefully-designed references to the exhibits, CHESS follows a plot-based approach, where the story authors (curators, museum staff and exhibition designers) write and produce stories around pre-selected museum themes.

Similarly to the making of a movie, our approach to interactive story creation includes four main phases, namely scripting, staging, producing and editing. During scripting, the author chooses the main concepts and intervening elements, sketches the plot, and writes the narrative text. In staging, the author associates parts of the script with exhibits, paths and other spots in the physical museum space. Then, a set of multimedia resources is produced for the staged script, including audio-visual material, interactive images, games, quizzes, augmented reality models, and other illustrative applications. Finally, the author edits, selects, and orders the multimedia digital resources to implement the final script into a storytelling experience.

The CHESS experience is a unique non-linear combination of the story presented through the terminal on the mobile device used, the visitor’s actions, the exhibits in the cultural heritage site, as well as the surrounding environment itself. When the visitor experiences the story on-site, she is subjected to five interlinked “experience modes”: (a) walking from exhibit to exhibit, (b) observing an exhibit, (c) listening to narrations from the terminal, (d) interacting with the terminal to make choices, and (e) using the terminal in interactive activities such as games or AR. Obviously, the design of such experiences requires careful orchestration of different resources.

CHESS provides story authors with the CHESS Authoring Tool (CAT) that enables the design and implementation of interactive stories for the CHESS system. CAT is based on a rich storytelling data model which uses graph-based representations to denote the story structure, along with structured meta-data to semantically describe the graph entities. CAT also enables authors to edit, annotate and enrich 2D and 3D maps, to create narration, menus and QR-scanning activities as well as to import AR applications, sophisticated games, etc. During the visit, the story graphs authored are traversed by the Adaptive Storytelling Engine, which uses visitor and contextual data to appropriately adapt the visitor's experience. Tthe CHESS system follows an implicit profiling approach, implementing generic visitor tracking and dynamic profile update techniques, and refining the visitor profile as the experience progresses. The CHESS stories are delivered to visitors through the Mobile Experiencing System, i.e. the framework running the CHESS experience on a mobile platform. It is based on InstantAR App which is optimized to display 3D content and embeds a state-of-the-art AR engine.

CHESS also covers the pre and post visit parts of the experience, which are handled by the Web Experiencing System. For instance, prior to visiting a museum, the visitors may browse information about it, play digital games, etc., while after the visit they can access an overview of their experience in situ. An important part of the pre-visit experience is the completion of a short, interactive questionnaire, the CHESS Visitor Survey (CVS). This is how visitors are registered in the CHESS system and their answers are communicated to the Adaptive Storytelling Engine for initial profile elicitation.

The CHESS System is a research prototype that has been developed in the context of the CHESS. It aims to enrich museum visits through personalized interactive storytelling, by (re-)injecting the sense of discovery and wonder in the visitors' experience. It uses personalized information to create customized stories that guide visitors through a museum and employs mixed reality and pervasive games techniques, ranging from narrations to Augmented Reality (AR) on mobile devices.

Conditions of Use: 

Some of the CHESS software components are proprietary (such as the CAT) while others are open source (such as the Adaptive Storytelling Engine). The story content produced is proprietary to the corresponding museums.

Contact person: Maria Vayanou - vayanou@di.uoa.gr

Use Case: 

CHESS targets two “types” of users; visitors, who “consume” CHESS stories through their devices, and story authors, who design the experiences. Aspiring to replace the traditional set of exhibit-centric descriptions by story-centric cohesive narrations with carefully-designed references to the exhibits, CHESS follows a plot-based approach, where the story authors (curators, museum staff and exhibition designers) write and produce stories around pre-selected museum themes. Similarly to the making of a movie, our approach to interactive story creation includes four main phases, namely scripting, staging, producing and editing. During scripting, the author chooses the main concepts and intervening elements, sketches the plot, and writes the narrative text. In staging, the author associates parts of the script with exhibits, paths and other spots in the physical museum space. Then, a set of multimedia resources is produced for the staged script, including audio-visual material, interactive images, games, quizzes, augmented reality models, and other illustrative applications. Finally, the author edits, selects, and orders the multimedia digital resources to implement the final script into a storytelling experience.

Made by: