Creatively Authoring Blue House Memories
2024 | Visualization Research Centre, HKBU
Project Overview
This project leverages immersive technology to transform the historic Blue House in Wan Chai into a living narrative. Residents and visitors collaboratively edit panoramic images within a 360-degree visualization system, preserving shared and personal histories. By integrating generative AI, the project encourages intergenerational storytelling, using creative editing to deepen the visual narrative and prototype tools for enhancing Hong Kong’s cultural heritage imagery in an immersive 360° theatre.
The project unfolds in three stages. It begins with ethnographic engagement, where discussions with residents identify the Blue House’s cultural significance. Next, participants edit images to infuse personal memories, adding resonance to the visuals. Finally, the AI model is refined based on participant feedback, ensuring it recognizes and reflects the project’s historical context.
Technical Narrative
Within a theatre setting, the 360° prototype offers a high-resolution, interactive experience that showcases both original heritage visuals and newly created images. Spoken prompts convert to text for AI-driven imagery generation, while gesture-tracking allows participants to select areas for editing. The adaptive AI responds to multi-user inputs, enabling collaborative shaping of the narrative.
In the final immersive experience, visitors explore the Blue House through a panoramic environment where they can add new objects that overlay existing visuals. This integration of past and present deepens engagement, creating a layered narrative that vividly brings the heritage site to life.
第一部分：專案概述
本專案運用沉浸式技術將位於灣仔的歷史建築藍屋轉化為一個活生生的敘事場景。居民和訪客可在 360 度視覺系統中共同編輯全景影像，保存共同的及個人歷史。透過生成式 AI 的整合，專案鼓勵跨世代的敘事，以創意編輯來深化視覺敘事，並創建工具原型來提升香港文化遺產影像於沉浸式 360° 劇場中的呈現效果。
專案分為三個階段。首先是以民族志為基礎的互動，通過與居民的對話確認藍屋的文化重要性。接下來，參與者編輯影像，融入個人回憶，增添視覺上的共鳴。最後，基於參與者的反饋進行 AI 模型的改良，確保其能識別並反映專案的歷史脈絡。
第二部分：技術敘述
在劇場環境中，360° 原型提供了一個高解析度的互動體驗，展示了原始文化遺產影像與新創影像。語音提示會轉換為文本以生成 AI 影像，同時手勢追蹤功能使參與者能選擇編輯區域。自適應 AI 回應多用戶的輸入，使敘事能夠協同塑造。
在最終的沉浸式體驗中，訪客可在全景環境中探索藍屋，並添加新物件覆蓋於現有影像之上。此過程將過去與現在融合，加深參與感，創造出層次豐富的敘事，生動地再現這座文化遺址。
Principal Investigators: Marty MILLER, Ray LC
Partner: …
Date: 2024 - ongoing
Team: Grace ZHANG (Research Assistant), Stephen WONG (Unity programmer), Zhiyuan ZHANG (Programmer, CS student)
Hardware: Controller, Projectors, Microphone, Workstation
Software: Unity, Fooocus LLM, Segment Anything LLM, Whisper, Low-rank adaptation (LoRAs)
Innovation and Technology Fund, HKSAR
