Pong to Pokémon: Evaluating the Visitor Experience of the Bullock Texas State History Museum

Pong to Pokémon: The Evolution of Electronic Gaming was the Bullock Texas State History Museum’s first foray into designing tech-heavy exhibits and immersive experiences in several years. For my Master's capstone project, I worked with the Director of Education and the Digital Media Team to identify and prioritize research objectives for exhibit evaluation in the museum.

Project Conception–On a trip to the Bullock with my Issues in Museum Studies class in the spring of 2017, I overheard that the museum was preparing for an exhibit on the history of video games to launch in the summer. As an avid gamer, I jumped at the chance to combine three of my passions: user research, video games, and museum studies. After discussing the lack of resources for in-house evaluation with the Director of Education, I proposed my capstone project on the spot.

Problem Statement–After several years of hosting touring exhibits with digital and immersive components, the Department of Education and the Digital Media Team at the Bullock wanted to incorporate more media and technology into the exhibits they curated in house. They had successfully proposed Pong to Pokémon, a tech-heavy exhibit which would allow them to experiment with a variety of media and digital interactives. Unfortunately, they had few resources for determining the effectiveness of their designs from a visitor's perspective before and after launch. Furthermore, without data supporting their efforts, it would be difficult to convince key stakeholders of the viability of their approach in the future.

My Solution–In a consulting role during the fall of 2017, I created an evaluation metric and conducted a thorough observational survey and series of interviews to evaluate several aspects of the Pong to Pokémon exhibit and whether it was achieving established engagement and learning objectives. My research yielded key insights about the space and the ways visitors interact within it, and my recommendations will be instrumental to future exhibit design at the Bullock. My findings concerning particular aspects of the exhibit, such as the arcade soundscape, have encouraged the curatorial and exhibit design staff that not only can they experiment with tech-heavy and immersive experiences in the future, but that these can be successful and effective aspects of the exhibit.

Objectives

To investigate visitors’ engagement with and the viability of the various media and interactives in the exhibit:

• Soundscape in the Arcade
• Videos and Infographic
• Music Trivia Interactive/Game

To explore visitors’ awareness and understanding of the following:

• Organization of the Exhibit
• Engagement with Real Artifacts ("Authenticity")

Methods

2 Months of Observation–After my initial meeting with the Director of Education and the Digital Media Team, I determined that observing visitors in the space would be helpful in establishing a basic understanding of how they explore the exhibit and also identifying additional areas for exploration for later interviews. In the meeting, a member of the Digital Media team mentioned concerns about the placement of certain interactive displays. As the space has three entrances/exits (which is unusual for an exhibit), they weren't sure which path visitors usually take through it. In order to address these concerns, I decided to track the following data to enable the team to later map visitors' journeys through the exhibit space:

• Direction Traveled Through Exhibit
• Entrance/Exit Used
• Media Interaction and Dwell
• Engagement Behaviors

15 Intercept Interviews–While discussing the goals of the exhibit with the Digital Media Team and the Director of Education, we identified several areas of focus for my research. As many of these areas related to how the exhibit facilitated learning, I decided that the best method to explore them would be through a series of interviews with both individuals and groups. As visitors' time in the museum is limited, I decided to keep these interviews short (15 minutes) to explore visitors' immediate takeaways from the exhibit. I used findings from my observations to inform the wording of my interview instrument and further explore the following:

• Most Impactful Artifact or Display
• Soundscape Awareness
• Authenticity
• Organization of the Exhibit
• Game Recommendation ("If there is one game you wish was included in the exhibit, what would it be and why?")

Key Findings

Exhibit layout not optimized for flow and dwell–This exhibit space is ring-shaped with three entrances and exits. During my observations and interviews, I noted that the most common path visitors take through the exhibit starts from the gallery entrance and moves clockwise through the exhibit. This should help with placement of both key interactives and museum labels, as only 2/3 of interview participants noticed an infographic that the Digital Media Team spent significant time designing, and only 1/3 of the participants fully interacted with it as a result of its placement.

Instructions not utilized–In my interviews, several groups mentioned struggling with games they had never used or that were set up differently due to an emulated environment. Moving forward, more testing will be needed for instructions for gameplay/interactives and media in future exhibits.

Challenges

• The size and shape of exhibit made observation of groups difficult, especially during Free First Sundays, when the exhibit was particularly crowded. I had to modify my observation protocol accordingly.
• It was difficult to convince visitors to agree to a 15 minute intercept interview when their time in the museum was so limited, even with museum vouchers as incentives.
• In the interviews, visitors had trouble estimating how much time they spent attending to any one aspect of an exhibit. This made self-reported data concerning dwell inconsistent and unreliable. I had to rely on my data from the observations for my final analysis and report.
• As most participants reported that they did not watch the videos in the exhibit, my interviews did not yield much data concerning engagement with the short films in the exhibit.

Next Steps

• The Digital Media Team has decided to use data from my research to start journey mapping as part of their exhibit design process for this particular exhibit space. They have also mentioned incorporating other UX principles into their designs for other spaces.
• I also drafted a survey for the Visitors Services staff to implement through the end of the exhibit to further explore the viability of the instruction panels for the interactives and how videos in the exhibit facilitate learning.

Outcomes

• The Bullock has recommitted itself to systematic evaluation for their exhibits and programs. My evaluation metrics, interview instrument, and observational survey will serve as a baseline for later evaluation models in tech-heavy exhibits at the museum.
• My evaluation metrics, data, and final report have enabled the Bullock to apply for tech-related awards for museum engagement. They were recently awarded with an Honorable Mention at the 2018 MUSE Media & Technology Awards for their music trivia interactive/game.
• My findings concerning certain aspects of the exhibit, such as the soundscape, have empowered the museum to move forward with immersive experiences. Several installations and 3 soundscapes and have been added to the museum since my project ended in December.