ChatterBot

2024 | Visualization Research Centre, HKBU

VISIT THE PLACES YOU HAVE DREAMED ABOUT
BEFORE YOU FLY. FLY THERE.
THE MOST IMMERSIVE 3D EXPERIENCE EVER.
VISIT UNSECO WORLD HERITAGE SITES ACROSS ASIA
DISCOVER THE BEAUTY OF CHATTER BOX

Project Overview
ChatterBot is an interactive art project that harnesses advanced AI technologies, including language models, speech-to-text, text-to-speech, and text-to-image systems. This project invites visitors to engage in verbal interactions with three groups of virtual entities, creating an immersive experience where these entities serve as dynamic embodiments of AI.

In a 360-degree interactive environment, three distinct groups of virtual entities are symmetrically positioned, each occupying a designated area of the panoramic view. This layout enables viewers to interact with multiple perspectives simultaneously, enhancing their immersive experience. Each entity appears as a swirling cloud of letters that dynamically shift as the chatbot speaks, with unique font styles reflecting their individual personalities. Voices vary in gender and accent, enriching the conversational diversity. Pre-set prompts guide each group’s discussions, establishing a framework for structured yet flexible dialogue.

Technical Interaction Narrative
Visitors use spatially tracked iPhones, which monitor their position and movement to facilitate real-time interaction. As they explore the environment, the system responds dynamically, allowing them to unlock new content and engage with AI-generated comic visuals, seamlessly blending the physical and digital spaces. When visitors approach a chatbot group with a mobile device, the conversation volume increases to a normal speech level. A microphone icon on the screen invites viewers to join in; when the icon turns green, it signals that the visitor can speak. The chatbots quickly adapt their responses, incorporating visitor input into the ongoing dialogue.

Above each chatbot’s animated font cloud, a graphic ‘bubble’ appears, mimicking the comic strip style. These bubbles display images generated in real time, responding directly to the chatbots’ spoken words. Rendered in black-and-white, these visuals reinforce the project’s comic strip aesthetic and deepen the interactive experience.

第一部分:項目概述

ChatterBot 是一個互動藝術項目,運用了先進的人工智能技術,包括語言模型、語音轉文字、文字轉語音以及文字轉圖像系統。該項目邀請訪客與三組虛擬實體進行語言互動,創造出一個沉浸式的體驗,讓這些實體成為AI的動態化身。

在360度的可視化環境中,三組不同的虛擬實體對稱地分佈在全景視圖中的特定區域,讓觀眾能同時與多個視角互動,增強沉浸感。每個實體以一團不斷變化的字母雲呈現,並隨著機器人說話而動態變化,不同的字體樣式也表達了各自的獨特個性。每個實體的聲音在性別和口音上各有不同,豐富了對話的多樣性。每組聊天機器人都有預設的對話提示,引導互動過程,構建了結構化而靈活的對話框架。

第二部分:技術互動敘述

訪客可使用空間追蹤的iPhone,這些設備可監測其位置和動作,以實現實時互動。隨著訪客探索環境,系統會動態回應,讓他們能解鎖新內容並與AI生成的漫畫圖像互動,從而將物理與數位空間無縫結合。當訪客接近某組聊天機器人時,該組的對話音量會增加至正常語音水平。手機屏幕上顯示一個白色麥克風圖標,訪客可輕觸以加入對話;當麥克風變綠時,代表訪客可以說話。此時,聊天機器人迅速適應並回應訪客的輸入,並將其意見平滑地融入進行中的對話。

在每個聊天機器人的字母雲上方,有一個仿漫畫對話框的「氣泡」,其中顯示的圖像會根據聊天機器人的語言內容即時生成。這些圖像以黑白漫畫風格呈現,強化了項目的互動性,並與漫畫風格的視覺效果相得益彰。

Principal Investigator: Jeffrey SHAW

Partner: 

Date: 2024

Team: Niko VOLZOW (software developer), Ching LEE

Hardware: nViz with 29.6 surround sound system, Projectors, HTC Vive tracker, iPhone

Software: Unity 3D + custom coding, Stable Diffusion, ChatGPT, MaxMSP, Dall-E, Microsoft Azure

Innovation and Technology Fund, HKSAR