VRLA Mixed Reality Easter Egg Hunt

  An unlikely project that became the highlight of VRLA 2017
 

Intro

Objective

Our objective was to create a mass-scale mixed reality Easter egg hunt using the HoloLens.
 

Inception

VRLA co-founder, John Root, originally pitched the idea to Microsoft. Microsoft then reached out to AfterNow, Microsoft's HoloLens agency partners, for help with this project.

The original concept looked like this:
Mindcraft  

Team

Experience design & development:
John Root
Philippe Lewicki
Rafai Eddy
Mike Murdock
Nathan Fulton
Ralph Barbagallo
Bryan Truong
Fon
Adam Tuliper

Set design:
FonCo

To run the experience:
Philippe Lewicki
Rafai Eddy
Jesse Vander Does
Bryan Truong
Kat
Josh
Vanessa
Kim
Diana
Rachel

Execution

The project was green-lit less than 4 weeks before the expo. Without time to spare, all the phases described below ran in parallel, which created a fair amount of synchronization costs.  

User Experience Process

Envisioning

Envisioning is the first thing we do in all our projects. The process is structured to get everyone on the same page by collaborating to creating parameters, generate a ton of ideas and ultimately select the best ones.
We have a well structured Envisioning process its flexible in duration between 1 and 5 days.

Given the time constraints of this project, we decided to do a quick :
 

Key Outcome From the Envisioning

  Goals we set:

  • The experience exceeds the customer's expectations
  • The experience creates a lot of buzz, i.e., have the press talking positively about it
  • Get 500 people through the experience

Our key constraints:

  • No gestures
  • No wifi
  • Make users forget about the field of view

The MVP will start with:

  • Ambient butterflies and birds
  • An animated rabbit
  • A number of eggs to be found in a limited time  

Demo-booth User Flow

We needed to outline the capacity, the movement and timing of people through the booth early on so we could get the construction of the set underway. Users needed to be comfortable waiting, and then we needed to figure out how to fit them with the device.

Key steps in the user flow included:

  • Standing in a long line outside the booth
  • Measuring IPD (inter-pupillary distance)
  • Getting the HoloLens securely fitted on
  • Receiving a brief introduction to the experience
  • Walking through a 3-minute experience (finding and opening 5 eggs)
  • Taking a group picture
  • Sharing email and providing experience feedback

Key criteria we needed to accommodate:

  • We wanted people to line up at the rear of the booth to make sure the view of the booth wasn't obstructed from passers by.
  • We needed to make sure that all HoloLens devices are at the booth at all times.
  • It was important to have space for visitors, press and staff to mingle and fill out surveys.
  • We needed to do as much as we can to make sure the HoloLenses didn't lose tracking. This ultimately meant taking down some walls between the prep area and the experience so the HoloLenses don't need to transition into a different space.

We sketched a few possibilities...

3D booth designed Booth with side open



  ...and ultimately settled on the one below as it checked off all the requirements.
Final userflow

To make the flow possible, we had to rotate the entire booth 90 degrees in the convention center.

To provide an efficient and enjoyable experience, timing and prepping the complete user flow on set was critical. We calculated it was optimal to send 4 visitors into the experience at a time and the 'Hunt' itself needed to be under 4 minutes in length. This meant every 4 visitors would spend a total of approximately 10 minutes to go through the setup, the 'Hunt' and the photo booth.

Users Test Different Aesthetics and Styles
Choosing the aesthetic theme of virtual elements in the experience was also important. The set had a specific look resembling a somewhat retro, Super Mario 64 feel, and we took some time deciding to either go with a realistic virtual theme which stood out more on set, or an angular/low-poly cartoon theme which matched the set.

We put a few sample models together and created two test scenes to switch between the different characters. After getting opinions from testers as well as internal team members, we decided to go with the matching cartoony theme.

In-experience game mechanics

We believe in user testing, and we put a lot of our assumptions to the test by seeking validation and feedback from end users.

Is the Egg Hunt fun?
Validating the core game-mechanic was easy to overlook in the midst of a very busy project, but we needed to find out if finding virtual eggs in the user's space was exciting. So, during our mixed reality meet-up in March, we put together a very simple HoloLens demo where we placed a few eggs in the room that opened when users gazed on it.

After over a dozen tests, we learned that the gaze-based game interaction worked but the game itself lacked in exciting users. When we introduced a timer, we were able to increase the level of urgency, but there was a larger void.

We needed to add more meaning to this experience and for that, we looked back at our documentation of ideas from our previous Envisioning process. It was clear to us that we needed somehow incrementally build tension as users progressed through the experience and provide a finale to end it all.

The Story

We had a plethora of ideas. For example, having animated cartoon characters that ask each visitor for help in retrieving their stolen eggs from an evil villain snake; or making a competition among the 4 visitors to collect the most eggs. For many of these ideas to work as well as we would have liked, we needed more time.

Also, for a lot of us, some of our most magical experiences have been We experimented with the concept of, “underworld”. The idea was for the big reveal to be a form of X-Ray vision. Suddenly, visitors would be able to see inside the trees . The trees would be full of little rooms for squirrels or rabbits.  This would require us to turn off spatial mapping and have a good scan of the set to create holes in. There was discussion to get a LIDAR scan, but nothing was locked in.  So we pushed that idea aside.
  The important part was to make sure we came up with a reliable way to guide the visitors through the experience in a timely fashion. Since we already had ambient animals as part of the experience, we decided to attempt at creating a timed sequence of different animals that guide users to each egg. Although it's not a shared experience, meaning each user would be opening their own eggs at different times, we could start all four user experiences simultaneously so eggs and animals appeared in the same places and at the same time for everyone.

So we did our second round of tests using the timed logic.

The great thing about the timed sequence was that it mimics the attributes of a shared experience. When the dog barked, everyone heard it at the same time and coming from the same place on set.

We used a butterfly, dog, rabbit and a dragon, all rigged with basic animations. We additionally created and recorded a narration to guide the users during the 3 minute experience.

Below is the first animation path we tested.

Animals path

It ended up being too long, so we removed an egg and accelerating the animations to fit our time constraints:

3 animals path

At that point, we started testing the timing on set (which was still being built).:
 

Overall, users responded positively to this flow. The timed animal guides combined with the narration tended to create a more cohesive response from users. Most users bunched up together and walked through the set which has its perks in providing a sense of purpose but it simultaneously took away from the element of discovery.

The Final Experience

Due to time constraints, we settled on a good medium for the final experience. We minimized the guidance to be handled largely by the narration and decided to spend the rest of our resources working on the ambient animals and objects in the scene for visitors to explore and be mesmerized by.

In the end, a dragon stole the show. As a visitor opened one of the eggs (which we strategically placed so they would most likely open it towards the end of their hunt), a dragon emerges out of the egg and flies dynamically around the 18 ft. tall tree in the center of the set. For the finale, fireworks shot off in the distance as user all gathered around for their photo booth shot as the experience concluded.

Modeling and Animations

It was important for us to make the egg-opening animation awesome as it was the one interaction every user repeats throughout the experience on multiple occasions. Ultimately, all eggs looked like large, regal gems that opened up like a puzzle and sent a colorful shockwave into it's surrounding landscape. It was beautiful. Other large modeling and animation sequences included the dragon flight, the bunnies fleeing from the set as users got close to them and flowers blooming as visitors gazed at them within proximity. Although we didn't change the animations in the scene a lot, its fair to say it took a fair amount of work and on-site testing to get the look right.  

Building the set

Coding the Experience (lets get more technical here)

Coding tests
We had code tests for the story and experience multiple times per week. Each experience needed to get coded in order to perform these tests, so we had a full-time developer devoted to writing tests and used unfinished assets and assets purchased on the asset store at times when necessary.

Coding the final optimized version
As mentioned earlier, it took a good chunk of out time to get all the final animation and lights to look and feel right. All of that without exceeding the technical limits of the HoloLens.

We tested the limit by loading as many assets into the experience as it could handle and looked for the threshold where we'd begin to see a significant drop in frame rate. We then made sure not to hit that threshold to steer away from any possibilities of performance issues during the experience.

Photo booth

The photo booth was a last minute addition to the experience and it was something we had in our Envisioning outcome as a stretch goal. The closer we moved towards the event date, the more we felt it needed to be in the experience to provide closure.

The idea was to offer participants a group photo at the end of the experience including all of the virtual elements in egg hunt.

It turned out to be a neat concept as we provided participants with something they can share on their social channels.

VrScout photoboot

We created a small client browser based application that would grab the photo taken from a HoloLens using the portal API.

The attack tweet  

Running the Experience

Building the experience was one part, running the experience was a whole other beast. The HoloLens is designed for the workplace, so designing an entertaining consumer experience was new and challenging.
 

Requirements

  • Clean the HoloLens for each use
  • Keep count of all HoloLens throughout the expo
  • Personalized IPD measurements for every visitor
  • 3-minutes limit to setup & brief 4 to 5 users
  • Keep the HoloLenses charged
  • Confirm the spatial mapping is aligned on each device, per use

Staff roles on demo floor

We defined 5 staff roles to run a well-oiled demo station:

1. IPD measurement
The first staff role was to stand at the end of the line and measure everyone's IPD, take note of them and hand it off to the tech staff. We regularly had two member out doing this as it was a good opportunity to answer questions about the experience and generally get the participants hopes up before entering the experience.

We originally used the Kinect IPD. It was less intrusive and really but it took too much time per user, so we scrapped it and went manual.
< picture of the kinect IPD >

2. Briefing & fitting
Four people (up to six at times) got sent into the setup area with their IPD measurements where the briefing & fitting staff assisted in fitting the uniquely measured HoloLenses onto every visitor's head. In order to assure the alignment wasn't off, we placed a green virtual butterfly on one of the walls which we asked them to confirm. If the butterfly wasn't there, we had to relaunch the application for the visitor.

Once all the devices were fitted securely on everyone's head, the staff member stands in front of them to give an introduction to the experience and makes sure the previous group of participants are donewith their on-set experience before instructing the next group to walk onto the set.

As soon as the group walks onto the set, the next four participants were pulled into the setup area.

3. Tech
The tech attendant was in charge or starting the application on all devices and customize the IPD for every participant. As soon as the measurements came into the setup area, they were in charge of plugging the each HoloLens into the laptop to set the IPD and place it on the table with the corresponding visitor's name tag on it so it was ready to be taken and fitted on by briefing & fitting.

4. Device maintenance
We had a total of 20 HoloLenses and it was crucial to make sure none of them were out of sight at anytime during the expo. We regularly had 2 staff members at a time for device maintenance as one needs to keep an eye on the devices on set and collect all of them when every group was done, while the other was cleaning each device after every use and made sure the devices were on a smooth rotation in order to maintain adequate battery life.

Hololens

5. Photo booth
The photo booth staff posted up next to a laptop right outside of the set area in an ideal location to shoot the end photo from. As a group of participants were ending the experience, this staff member took a group shot of all of them using his/her HoloLens and gathered all of the participants around the computer to captured their email addresses.

When participants were done inputting their email addresses they all got delicious dark chocolate bars and some willing members took a survey.

Chocolate

Conclusion

  This project was indeed an awesome learning experience and a great success. There is a lot of work that goes into running a large demo booth smoothly and now we are more confident to be able to create ones of similar scale.

Check out our Case Study for all the details.