Ghasper - The Campus Ghost

A Playful Interactive Halloween Installation

Design Research
Design Programming
Interaction Design

How can we Effectively Design Playful Interactions with Hybrid-Resolution Media Facades?

Jump to Final Design

Context:
Master's Degree Capstone/Thesis - I worked with an academic supervisor to conduct design research into a topic of choice. The outcome was a design project along with a research paper detailing a framework for effectively designing playful interactions with hybrid resolution media facades

My Role:
Solo research project - Researcher, Designer and Programmer

Academic supervisor:
Luke Hespanol

Background

The Footbridge Gallery

The Footbridge Gallery, located at the University of Sydney, is a digital placemaking area with embedded light and sound technologies opened in 2019. It is the brainchild of Dr Luke Hespanhol and Associate Professor Martin Tomitsch from the University’s School of Architecture, Design and Planning.

For this project I was to create a design and conduct research into a topic of interest using the technologies at The Footbridge Gallery which consisted of a large projection on the building (High resolution) and programmable LED strips (Low resolution)

Literature Review

Background Research

Understanding what has been done before

The first step was conducting a literature review through reading numerous academic journal papers, articles and websites. The aim was to gather information and analyse it to gain an idea of what has been done in this subject area and the type of knowledge that has been created. Since “Research is defined as the creation of new knowledge and/or the use of existing knowledge in a new and creative way so as to generate new concepts, methodologies, inventions and understandings” (Australian Research Council), it was an important step to find a gap in which my research could be directed and scoped to fit into this field in a way that would generate new knowledge.

I then synthesised the information organised it into these topics:

  • Smart cities - Data about the city and its citizens are collected through pervasive technologies such as wearables and social media to build a city that is responsive and adaptive to the needs of citizens, with the main focus being 'efficiency’.
  • Playful cities - emerged from this opposition to this focus on "efficiency" and control to create an ideal "city as a machine" . Popular concerns include control of citizens, data privacy and concerns that smart cities tend to be driven by rules that focus on productivity and codifying behaviour enforcing a social norm for citizens to be.
  • Designing for high resolution - many studies have been conducted into effective techniques and strategies to designing high resolution screens such as mobile phones, projections and large interactive screens
  • Designing for Low resolution - although less studied that high resolution, content and interaction design for low resolution has been studied.

However, with the placement of the high and low resolution mediums being on top of each other at the Footbridge Gallery I decided to look into these technologies being combined together. The current designs that have been at the Footbridge Gallery were also only either only LEDS or only Projection. Thus this became the focus of my project and termed it "Hybrid Resolution".

The Design Knowledge Gap

I also looked at the design precedents and knowledge which could be summarised into this table below which shows a clear knowledge gap that I could aim to address in my research.

I tried to find studies and existing designs that involved hybrid resolution, but there were very limited amount of studies. With that in mind I decided to base my study on designing playful interactions with Hybrid resolution Media facades - filling in that design knowledge gap

What type of Playful Interaction?

Through further research I came across the concept of personifying or animating things in the world. Associating spirits with objects or personifying them has been recognised by Sociologists as a concept widespread among the peoples of the world (Bell, 1997). A modern-day example of this is a wedding ring given by the spouse that would feel different to a replacement that looks the exact same. Bell argues that in the same way as giving souls to objects, we give space social meaning.

Along with the fact that Halloween day was near the project deadline, I decided to create an interactive ghost character with the technologies in the public space to conduct the research.

Define Scope

Aim of the Research Project

There were two components to this Research Project:

A) Design Project

Design and implement a playful interactive character in the public urban space

B) Academic Research

Investigate design challenges and strategies for creating a playful interaction with hybrid-resolution media façades as well as people's perceptions towards such an interface. In particular, the research was to address these questions:

  1. How can high- and low- resolution media façades be combined effectively to design a playful interaction with hybrid-resolution media façades?
  2. What are the user perceptions towards an interactive character media installation?
  3. How does audience behaviour differ towards the installation when coming to the area expecting to interact with it (e.g. an event) vs unintentionally interacting with it (part of urban life)?

The two components were intertwined in that I would be creating the design while conducting research into designing it.

Methodology

Research In and Through Design

I adopted a methodology based on an approach called "Research In and Through Design" where the interaction design researchers are engaged in "the development of an interactive façade in order to generate insights into how to plan and carry out design for this type of interactive systems” (Dalsgaard, 2010, p.1).

This was chosen because there is little established knowledge on the design process for this type of interface (Hybrid-Resolution media facades). The characteristic of this methodology is that it has two parts (seen as 2 loops in diagram):

Part 1 - First series of design experiments to gain insight into how to conduct the research and design aspects to consider for the novel interface

Part 2 - Insights from part 1 can then be used to produce a final design through design iterations while continuing to conduct research

The Study Method

Translating the above process into my project, I decided the two parts could be as follows:

Part 1 : Non-Hybrid-Resolution Design Experiments

3 Studies and a Baseline study looking at the merits of Low resolution and High resolution designs separately

Part 2: Hybrid-Resolution Design Iterations

5 studies conducted that involved iterations of design concepts that combined the High and Low resolutions

What happens in each Study?

Each Study was 1 hour in duration consisting of two 30 minutes sessions:

  • Session A – running Design 1
  • Session B – running Design 2

Each study either differed in experiment variables or designs. For each session we kept “interaction counts" by using a tool I developed to tally the number of people who would:

  • Pass - walked across the installation without looking up at it
  • Glance - looked up when passing by for a short time, without stopping.
  • Interact - stopped to look at the installation or perform actions beyond glancing e.g. taking photos or pointing

Observations from a distance and semi-structured interviews were also conducted during the sessions.

Technologies Used

  • The hardware infrastructure - which includes the LEDs attached to the building facade and Projector attached to building opposite the wall were setup by my supervisor the year before
  • The interface to interact with the hardware - was set up by my supervisor including the use of screen sharing and ARTNET coding libraries
  • LED light pattern - I used the Processing Java library to code the LEDs with the base code from interface described above
  • Projector graphic - I used the Processing Java library to code the animations, control and the OpenKinect Library to interface with the xbox Kinect for detecting passers-by
  • The Interaction counter tool - coded with Processing Java Library

Part 1

Non-Hybrid Design Experiments

  • For each study in Part 1, we used the first 30 minutes to show a Low-res design (using only the LEDs) and then a high-resolution design (using only the projection). Both had similar designs so that I could compare the merits and disadvantages of each.
  • These initial studies also allowed me to test concepts and learn about designing for this particular set of technologies. It gave me the knowledge and practice to implement further concepts and ideas.
  • These were also all "Wizard of Oz" I.e. the eyes "following" people were manually controlled by my supervisor while I conducted the interviews.

Curiosity Objects: Additional decorations and items aimed to attract curiosity and attention - I stuck paper ghost cutouts near the installation and a put cloth over a statue with two paper eyes

Summary of Findings:

  • The fact that the LED lights and projection are a permanent installation that has been there for a year and turned on every night meant that people didn't pay attention to a different design - particularly, the low-resolution ghost designs were not recognised to be different from the usual abstract colours
  • The large majority did not suspect the eyes to be "following" them
  • While "eyes" were thought to be creepy, adding a face/body to it turned it into a character that was mostly described as "Cute"
  • Adding the curiosity objects and sound attracted much more attention and passers-by took interest because they thought "It's only today, it's not usually there"
  • The baseline study saw people to pay attention for longer, leading to a hypothesis that the changing nature of it (different designs throughout the 30mins) compared to the ghost (one design throughout 30 mins) retained more attention
Show Design Experiments DetailsHide Design Experiments Details

Part 2

Hybrid-Resolution Design Concepts

Part 2 involved creating and iterating on a Hybrid Resolution Design and really started to look what the hybrid-resolution ghost character would be like. From part one findings, two design challenges were chosen as the basis for two Hybrid-resolution design concepts:

Concept 1: Light Shadow

Challenge:
No one perceived that it could be interactive

Concept:
Using Low-Res as visual feedback for interaction to support the High-Res image.

As people walk across, LEDs will light up to form a “shadow” of the user as they walk across. With multiple people, the person/shadow being “tracked" will be coloured while the rest are white

Concept 2: Hybrid Ghost

Challenge:
Low-Resolution image was difficult to recognise

Concept:
Creating a continuous image using the High-Res image to help recognition of the Low-Res image.

The High-Res displays the “head” of the ghost whereas the Low-Res will be connected to show the “body” of the ghost. The eye pupils will follow people

Hybrid Resolution Design Iterations

I conducted 3 iterations through 5 Studies. For each study, I split the hour into 30 minutes each for the 2 concepts. Only Concept 1 (Light Shadow) used an X-box Kinect to detect motion, the second concept was kept as manually controlled but the Kinect was left in front of the installation.

Defining the Goal Experience

Although the main aim is to conduct research, I also focused on making it into a great experience. Thus from part one, I defined 2 types of passers-by to cater for:

Regular

  • Regularly walks past the area
  • Unamused by current installation as it is always there but does like it as "colourful lights"

First time

  • Walks past the area for the first time
  • Unfamiliar with the university and notices the installation, usually wondering if it is a permanent installation or temporary

I then also defined keywords to describe the experience I hope to create for them:

  • Fun - having no particular purpose other than bringing some joy
  • Engaging - people would interact with it, more than a glance
  • Surprise - to be recognised as unusual and out of people's expectations of the installation
  • Curiosity - create a sense of wonder and prompt people to question it

Creating the Experience

There were also study "pairs" where I used the same designs across two studies but the difference was whether I promoted the study as an event or not (through event posters around campus and social media) - to answer the third research question - Will behaviours and perception differ when people come to the area knowing about the Installation?

Similar to Part 1, I conducted onsite semi-structured interviews and interaction counts were recorded. Findings were analysed and changes implemented for each iteration as seen below.

Summary of Design considerations and findings

  • I decided to keep the curiosity objects (ghost Stickers, ghost statue and the audio) from part 1 for most of the sessions as people noted that it added to the vibe and particularly the music was a large influence to attract attention - I decided on the "spooky magical" playlist over a more "haunted" playlist as the ghost was not a scary design
  • For concept 1 (Light shadow), having the light shadow position more accurately align to the position of the user (through technical iterations) greatly impacted understanding of it
  • For concept 2 (Hybrid ghost) the initial change from only the ghost eye pupils following passersby to having the whole ghost follow the passersby made a big difference to the understanding of the Low-resolution "body" being connected to the High resolution "head"
  • I then added colour to the design; red, blue and Green themes that changed at random intervals which retained attention. As one interviewee who stopped for a while explained “I was looking at the colours and I was like how many, is it just looping like the one thing?” and after being explained the concept they said “I want to keep watching it, because it’s not on a loop"
  • I also made the pupils change to a heart shape if people interacted with it - to further enhance the idea that some people said it could be like a companion - the change saw more people ask questions about it as if it was alive like “does it come out every night?” rather than previously (no heart pupil) “is this here every night?”. One stated that it showed “friendliness towards the people…if I come here to chill, I will feel like as if I have a companion nearby so I won't be lonely”.
  • The "ping" sounds that triggered when the eyes blinked were changed to a cute "pop" sound as people expressed previously that the"game" sound didn't fit with the friendly ghost theme.
  • As the studies drew closer to Halloween day, people started to guess that it was for halloween and recognise it as a ghost
Show Design Experiments DetailsHide Design Experiments Details

Final Design - Halloween Day

Ghasper The Campus Ghost

Video of Final Iterations

Outcomes

The RES Framework

Research Question 1: How can high- and low- resolution media façades be combined effectively to design a playful interaction with hybrid-resolution media façades?

Through synthesising the results after each study in terms of which design changes had the most impact, I derived three key considerations and corresponding strategies for designing hybrid-resolution media façades into the RES framework along with the 3Cs Strategy, which could inform future designs of hybrid-resolution installations. This framework and its sample application to this project can be seen below:

Research Question 2: What are the user perceptions towards an interactive character media installation?

Interviewee responses were very positive and showed that I was able to meet my experience goals:

  • Fun - was a keyword often used by interviewees when asked to describe it
  • Engaging - some tested out how "fast" they could be detected and engaged with it, many took photos
  • Surprise - it caught some people by surprise particularly when the colours and heart pupils were added. The curiosity objects added to this
  • Curiosity - Many questions were asked by interviewees and during interviews, few would stop by to listen in to my explanations of the installation. People also stopped to wonder what else there was.

A bonus effect was that people reported that it:

  • Gave a sense of Safety - "From a female point of view...so you are not walking in the dark, it would be unsafe if no one was here...It makes it seem like something is happening... adds liveliness"
  • Was like a companion - "I would be friends with it, its a lonely ghost...I would just stand in front of the sensor and it would just stare at me forever"
Research Question 3: How does audience behaviour differ towards the installation when coming to the area expecting to interact with it (e.g. an event) vs unintentionally interacting with it (part of urban life)?

Unfortunately, my attempt to promote the event was not very effective, due to challenges in affixing posters around the campus, I only have preliminary results for the difference in audience behaviours between the two scenarios.

Unintentional:

  • Often stopped late and went back to interact or keep walking while turning their heads
  • See interaction triggered by others in front of them which pigued curiosity
  • Questions they asked focused on whys and what the ghost is doing

Expecting:

  • Tended to linger longer and watched people
  • Came to take in the vibe
  • Questions they asked focused on what else they can do to trigger effects

Learnings

Conducting a Solo Research Project

  • There was a lot more freedom than I had expected- no particular methods or topics were set which meant really I had to plan everything. The guidance received was helpful, but nothing really dictated what I should do or even what to try to find out.
  • Although having a research plan is common sense, I was surprised at the amount of effort it involved and the importance of really scoping it through formulating research questions - it provides the direction
  • I learnt about "creating new knowledge" - the lack of guidelines and practices on designing with Hybrid-resolution facades meant I had to find out and think about it by myself on how to obtain that knowledge
  • Having a one on one supervisor was new to me where through this project I learnt to be comfortable asking about all the doubts and confusions I had as well as making sure I plan each meeting to not waste his time
  • I also learnt that documenting the research or design process is very helpful, both for others and for the future you. This includes taking pictures and videos of the work to be showcased to others or for you to remember certain decisions.

Conducting interviews in the wild

  • Conducting the interviews by walking up to random strangers got me out of my comfort zone - it provided me a chance to practice striking up conversations with strangers
  • At the start of the semester I had a lot of fears around being rejected from the interview requests, but it helped to remember that the worse that can happen is they say no. No big deal. Near the end, I could easily just walk up and say, “Hi can we have a chat for two minutes about that?” and not fear rejection.
  • Asking for voice recordings was also something I was afraid everyone would decline. But later on, due to the need to remember what people said, I asked and found that most people will say ok and it is ok if they decline.

Managing time commitments

  • With full time work, hobbies and other commitments, I learnt that managing time for this research project came down to two things: Priorities and Balance
  • Priorities - knowing that this is just one semester, I could figure out commitments that I am willing to pause for a while to focus on the priorities.
  • Balance - meant that of those commitments I made sure to keep the important ones as well as some leisure ones to keep me motivated.

Send a virtual wave 👋

peggyliu.uilyggep@gmail.com