Cross-Media Interaction in IoT-Enabled Environments: A Study of Cognitive Transformation

Liu, Shih-Ta; Hsu, Su-Chu. (2026). Cross-Media Interaction in IoT-Enabled Environments: A Study of Cognitive Transformations. 19th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA 2026), Island of Crete, Greece.

作者:Shih-Ta Liu, Su-Chu Hsu

摘要:Drawing on E. J. Gibson’s (1969) perceptual learning theory, this study explores how active participation initiates processes of cognitive transformation. Building on our patented IoT Cross-Media framework (2018), we examine data transmission and re-presentation across heterogeneous media. In 2025, we collaborated with three rural elementary schools to co-host the “Interactive Halloween Eerie Festival”, where an IoT-enabled interactive environment featuring multiple technological art installations was constructed. Using the interactive artwork “Taming the Finger Monster in the Black Forest” as a case study, this research investigates how participants’ learning behaviors and cognition transform within an IoT Cross-Media system. In the artwork, participants use finger gestures to generate personalized finger monsters that appear hidden among projected tree trunks on an interactive wall. Children use mobile phones to awaken, search for, and retrieve the monsters, while simultaneously responding to STEAM-related questions via a mobile application; their academic performance supports the monsters’ growth. Through qualitative observations and post-event interviews, this study analyzes cognitive transformation across three stages of interaction — creating, searching, and nurturing — ranging from embodied perception through finger-based creation, to emotional responsibility and risk-taking during exploration, and finally to caring for and nurturing digital life. The study further illustrates how these festival practices align with the United Nations Sustainable Development Goals’ vision of social service.

關鍵字:Cognitive Transformation, IoT Cross-Media, Interactive Learning, Halloween, University Social Responsibility (USR), Sustainable Development Goals (SDGs)

原始論文 DOI: 尚未發布(Springer CCIS 系列出版中,已於2026/7/15 PETRA 2026 Session G-1 正式發表)