Project: Night at the Museum 🏛

I’m working on a project for my studies with Udacity. The goal is to create a VR experience that demonstrates my research into a VR company. I’ll do a series of posts on this as I progress along the path to completion.

To start here’s my first crack at the initial documentation for the project:

Statement of purpose

The purpose of the "Night at the museum" application is educate the user on the HTC Vive VR solution. The experience is based on an exhibit style scene, which contains 5 "stations", each covering some aspect of the HTC Vive.

Persona

The target audience for this application is anyone interested in VR that would like to learn more about the HTC Vive VR solution. Here’s a basic persona from them:

S03e13_percival_1600x1200Name: Wade Watts
Age: 18
Role: Budding VR enthusiast  
VR Experience: Experience with google cardboard but limited beyond that
Quote: "VR is very exciting, I can't wait to learn more"
About this person: Wade has grown up with computers, game consoles, tablets and smart phones. He's a digital native that loves to escape into his devices. VR is the ultimate expression of that escape.

Concepts

I’ve started by looking at different museum and exhibit styles to select a something that fits the aesthetic I’m looking for. After looking at altogether too many museums (if that’s possible) I eventually settled on classic Greek style building with exhibits setup like sculptures on pedestals. I’m using the following images as inspiration for the design:

I like this style, it reminds me of some of my favourite museum tours, and I’ve also done some work in a previous project that shares a similar style. This should allow me to reuse some of the assets from that project, meaning I can move faster. Always important when you’re pressed for time.


Concept build

Having established a concept for the style of the experience a setup to make a concept build. Using the assets from previous work made putting something workable together relatively easy.

I added a terrain GameObject with a few mountains in the distance to give the scene a sense of depth. I dropped in a “temple” model that is a good fit for a museum and added a few items to flesh out the concept. For the actual stations I acquired an HTC Vive model from sketchfab user Eternal Realm.  I also kept an eye out for the scenes frame rate to make sure I wasn’t introducing anything at this early stage that had a big impact on performance. I still have a few more models to acquire but I’m happy with the direction the concept is heading in.

Screen Shot 2017-08-16 at 9.06.16 PM

A view from a potential starting position for the user.

Screen Shot 2017-08-16 at 9.06.29 PM

The users view approaching the entrance of the museum itself.

Screen Shot 2017-08-16 at 9.06.37 PM

A simple example of what a “station” may look like within the museum. A floating Vive headset over a stone platform.


Next steps

Now that I’ve got the basics in place the next thing on the agenda is the first round of user testing. I shouldn’t have to much trouble acquiring a suitable test subject to get things moving. I will be interesting to see how the scene changes and progresses over the course of development.

VR: Reshaping industries 🤔

As a bit of a thought experiment I’m taking a look at an industry and imagine a VR app might reshaping it.

Given my family’s always been in the construction industry (flooring), I thought it might be interesting to look into a segment of that market; medium to large scale commercial building construction.

To that end here’s a concept for an application that would help a site foreman and workers during a construction project to visualise the job to be done.

Statement of purpose

Create a virtual version of a construction site that shows the project at various stages of construction. Allowing the team to collectively understand what is to be done and how things should be at various stages. The application should also demonstrate hazards on site without exposing workers to those hazards.

Persona



Age: 30-65

Occupation: Site foreman

Name: Alex

A Quote:

"A 200 page job spec might be required reading, but no ones actually across it all. It would be great to be able to see the spec and show it to others to explain the detail"

2–3 sentences describing what motivates them:

Alex works long hours, and carries a lot of information and responsibility in their head. A key part of Alex's role is communication and having better tools to align everyone's activities on site are critical to a success project.

Staff safety is also a top priority. Giving people a clear understand of what's going on across the site keeps everyone safe.

Their experience level with VR:

Little to none

Q and A

Q: How accessible would each VR platform be to your target student in terms of price? Take into account location, age, and income.

A: The only platforms that are really accessible is Cardboard/Daydream or Gear VR style VR. A construction site can be a busy, dirty place. An expensive tethered full immersion rig really isn't practical on site.
Q: How interactive does your lesson need to be? For example, do I need to pick things up or could I get away with just looking at objects?

A: The user needs to be able to look at various objects, select them and have some details displayed to them. Movement would be required to view various scenes but does not require movement within a scene.
Q: How realistic do your visuals need to be in order to teach? For example, could I use 2D images and videos in a 3D Environment or do you need high poly 3D models.

A: images clear enough to show construction detail and highlight hazards is sufficient. I would also be useful in some cases to be able to show exploded views of constructions, but this could also be done via images or simple animations.
Q: Does my student need to feel like a participant in the experience or can they be a passive viewer? Could they be both?

A: Primarily they could be a passive viewer. The purpose of this app is to give the user a sense of what's going on, what needs to be done and what safety issues there are. There's little to no complex interaction required.
Q: Given the answers above, what are potential platforms you could use for your experience?

A: In this case I think any platform could work, providing the right conditions. For example a dedicated space would be required for a high immersion setup. Having said that it would be far more practical to use a mobile setup on a construction site.

VR course exercise 🤔

As part of my VR course, I’ve been asked to perform a bit of a thought experiment. The outline of this work is:

Imagine you want to make a VR Education Application. This could be an exploration of the solar system, looking at fine grain detail of the human body, or teaching somebody how to program, just to name a few topics. For this experiment, pick any topic you would like to teach. And don’t worry if it doesn’t make sense in VR, we will worry about that part later. Just something you are passionate about.

The requirements include defining a statement of purpose, a persona for the target audience, answer a few short questions and post it to Medium. Given I have my own blog I figured I’d also post it here:

Statement of purpose

To demonstrate the Vet teaching hospital facilities at Massey University to prospective students.

Persona

High school students

Age: 17–18 years old

Occupation, if any: High school student

Name: Sam

A Quote: “I love working with and caring for animals”

2–3 sentences describing what motivates them:

They love working with and caring for animals. Learning in the best environment possible is important to be effective at their jobs. They want to become excellent, leaders in their field, to have the greatest impact. They are likely unsure of their specialised, but would have some idea of the options available. They already know they care for animals but are likely unsure of what the day to days of being a Vet are, but will be highly motivated to learn.

Their experience level with VR

May have had some small exposure to VR and education events or possibly in school. Given its still early days in the tech itself, it’s unlikely they are highly experienced.

Q and A

Q: How accessible would each VR platform be to your target student in terms of price? Take into account location, age, and income.

A: Accessibility of the various system is wildly different.

Mobile
Most high schoolers, of this age, have smartphones and the means to acquire a cardboard headset. Also, headsets could be supplied by the provider via the post, or at events.

High immersion
Much less accessible. The most likely source for high immersion would be the PlayStation VR and it’s unlikely prospects would think to use this given its context for gaming. The Vive and Oculus don't have enough presence yet to be meaningful for this use.
Q: How interactive does your lesson need to be? For example, do I need to pick things up or could I get away with just looking at objects?

A: The experience is purely to show off a facility and teaching capabilities, so only basic interaction is required. Select objects for more details or to play videos/animations.
Q: How realistic do your visuals need to be in order to teach? For example, could I use 2D images and videos in a 3D Environment or do you need high poly 3D models?

A: 2D images and video would be ample to communicate what needs to be communicated. If in future there was a need to train in the use of equipment etc then a more immersive VR platform would be required.
Q: Does my student need to feel like a participant in the experience or can they be a passive viewer? Could they be both?

A: To a certain extent they could be both. Selecting options (pick a path style) or just following a set path. Having the option is likely a good idea given repeat users are likely to have something specific to see rather than the whole walkthrough. So it would be good to be able to skip to the parts they are most interested in.
Q: Given the answers above, what are potential platforms you could use for your experience?

A: Given the limited access to high-end VR hardware, mobile VR (aka cardboard/daydream/gearvr) are likely the best targets. It’s fair to say highly immersive VR would be richer, but given it would reach far fewer people it’s not really relevant for outreach activities like this. The fact the content could also be easily reused and delivered via the web is also a plus for mobile.

Future technologies

Future tech – Microsoft Research: NormalTouch and TextureTouch

A few questions to consider with regards to future technologies:

Q: How would Augmented Reality better help teach your experience?

A: In this case, it likely wouldn't be. Given we are attempting to educate student prospects on what the facilities are actually like at the University recreating that experience is better suited to VR. Augmented reality could potentially be used differently if you wanted to demo a particular part of the experience, such as show a horse on a large animal MRI machine, at an event. This could give prospects a real sense of scale relative to things around them.
Q: How could eye tracking help you better tailor your experience to your students?

A: Using eye tracking to understand the users focus within an experience could enhance our ability to provide depth of field. Making the experience more natural for your eyes. We could also use I tracking to monitor what users are looking at and use this information to improve the experience over time.
Q: How would better Haptics better teach your experience?

A: Introducing a tactile feel to the animals in the environment could provide a great sense of immersion and connection with the animals. The more real it seems the more convincing it is as a sales tool.
Q: How important is graphical fidelity to your experience?

A: Given the experience is likely relatively short users are unlikely to experience eye strain. Also, we are only attempting to highlight the environment the student will be in a demo the sophistication of the facilities. Given this is the case, it's not completely necessary for the graphic fidelity to be top of the line. Having said that, improved displays that take advantage of light field technologies would vastly improve player comfort and as a result will be essential and common place in the future.
Q: How critical is it that your target student receives this training within the next two years?

A: As the university expands internationally it's of increasing value to demonstrate to international prospects what facilities are on offer without them having the expense of physically travelling to New Zealand to see them first hand. Using VR allows them to get a sense of the place, helping them make more informed decisions.

Process: Puzzler 👨‍💻

Over the past few months I’ve been studying with Udacity to learn VR software development. The course has been great so far, so I thought I’d share a little of what I’ve been up to with a game called “Puzzler”.

About Puzzler

Puzzler is a simple VR experience for Google cardboard. Basically anyone with a phone and a $15 cardboard headset can give it a try. The game has a simple UI where the user is thrust into a dungeon, where 5 magic orbs present a puzzle. Successfully playing a single round of “Simon” gets the player out of the dungeon to victory.

Here’s a brief video of the “final” version of the game:

The process

Of course, I didn’t just wake up one morning and build this thing. There is quite a process to go through to create a good VR experience. As I mentioned on this blog before, a good process can make all the difference when you’re building for someone that isn’t you. Which, let’s face it, is almost always.

The approach taken for this work was no exception so let’s break it down a bit.

Statement of purpose

I started the process by creating a simple statement of purpose for the game:

Puzzler is a mobile VR app which gives new VR users a quick taste of VR via their existing smart phone. The entire experience should take no more than a few minutes and be accessible to most anyone that's physically able.

With this in mind, I selected the nearest available human to be a test subject and personify my ideal target audience. My 6-year-old daughter Emma was the winner on the day. When starting out the build, first I documented a little about her, so I had a clearly defined outline to work too. Here’s the persona for Emma:

Persona

Image:
Emma
Name: Emma
Age: 6
Role: Child
VR Experience: Has played a few simple VR games, but not many
Quote (that sums up their attitude): “Can I play with your phone thingy, it’s cool.”
About this person: Emma is an enthusiastic VR user, she loves to explore and have adventures. She’s young and enjoys content that's exciting and interesting, but not too scary or intimidating. She will ask lots of questions and enjoys the discovery. I think it's fair to say she has moxie.

With a clear purpose and audience/persona defined it was time to get going with an initial design and an “alpha” build of Puzzler.

Concept sketches

I started by creating a bunch of really nasty looking sketches for what puzzler might look like. And when I mean nasty I mean nasty.

Here are a few concepts I had for the game initially:

My thinking with the above designs was my audience is young. Given that, it was better to use a large simple UI that could be easily understood and a very simple scene design that wasn’t overly difficult to understand. Once I sketched out something I was happy with I then took the designs and built something in Unity.

User testing

With the first cut done it was time to get going with user testing. It’s so important to get an early version in front of your audience to test assumptions. My early tests were really all about determining the basics, like the scale of the scene in relation to the subject.

Basically, from there on it was a process of iterating on the project. A mix of making assumptions, asking questions, and testing it all as often as was practical. To give you some idea of the things I asked, here’s a few of the Q&A sessions I had with Emma:

User test 1

Me: How big do you think you are in this scene?
Emma: I feel a bit smaller, normally I think I’d be taller than that barrel. I feel little.
Me: What’s the mood or atmosphere of the environment?
Emma: A bit spooky, but I like the purple balls. Not too scary, there are no witches, I don’t like witches. It’s a bit bright for a dungeon though.
Me: Is there anything you find difficult to see, or anything visual you think could be improved?
Emma: No I can see everything, but I can’t hear anything should I be able to hear things?
Conclusion: Emma picked up on the spooky dungeon, she was feeling too small so I adjusted the player's height slightly. I moved the orbs to be closer to the player since Emma liked them so much. I also added some sound to the environment.

User test 2

Me: How big did you think you are in this scene?
Emma: I think it's right. I'm the right height.
Me: What’s the mood or atmosphere of the environment?
Emma: Spooky, the sounds are spooky, it sounds like night time. It’s a bit dark.
Me: Is there anything you find difficult to see, or anything visual you think could be improved?
Emma: The balls are too close to me. I feel like I’m going to bang into the balls.
Conclusion: Emma feels the right size now, but I think the scene is a bit too spooky for her now and the orbs are making for feeling a bit close. I’ll move the orbs to a different location and increase the lighting a bit so it's not so scary.
IMG_4518

Emma hard at work user testing

User test 3

Me: How do you feel about the scene
Emma: It looks cool, I like the lights, it feels dark outside and spooky inside, but warmer.
Me: Do you understand how to start the game
Emma: Yes I click the big “Start”
Me: Do you know how to play the game?
Emma: Yes it's like “Simon says”. I do what the puzzle does.
Me: Is there anything else?
Emma: The balls are in the way of the door. I have to crash into them when I win. The game is too hard for me.
Conclusion - The game has matured to the point Emma is happy with it and can play it well, she still isn’t happy with the placement of the orbs so I’ve moved them back and did a quick retest, she’s now happy. I also reduced the complexity of the game to 5 steps.

Breakdown of final deliverable

So, in the end, we’ve got a dungeon puzzler that isn’t too scary for a 6-year-old to play. Emma likes the game and can successfully play it.

The basic break down of the “final product” is:

  • The user is presented with a simple UI screen to start the game.
Screen Shot 2017-07-30 at 3.06.40 PM

The start screen as see from the Unity editor

  • On clicking start the player is moved into the dungeon where they are presented with 5 magic orbs. Emma’s feedback had a significant influence on orb placement and the feel of the dungeon.
  • The orbs chime and blink in a random sequence which the player must complete to “escape”.
Screen Shot 2017-07-30 at 3.05.49 PM

The magic orbs as seen from the Unity editor

  • If the player fails to repeat the sequence a “fail sound” plays, and the puzzle is repeated to give the player another chance. If they get it correct they’re moved out of the dungeon and are presented with congratulations and the ability to restart.

Conclusion

Over all, it was a fun educational exercise doing this project. Plenty of things to learn from a process perspective but also in Unity 3D development. The work has given me a few ideas for other projects to experiment with, and further solidified my view of user testing and rapid iterations in the development of a product.

Next steps

Next I think I’ll move on to a new project, but stay focused on something for that target audience. I think it would be of value to include a few more testers, including Emma’s brother and friends.

If you’re interested in playing around with the above project I’ve shared the source code via bitbucket. The other projects I’ve done via the Udacity course are also available on the Geekpulp bitbucket page.

VR design – Google Street View for iOS 🗺

Part of my Udcaity VR course involves doing a quick/light review of a Google cardboard app. So I figured I might as well post that here too.

VR App review – Google Street View – iOS

To call Google Street View for iOS a VR app feels like a bit of a stretch. Upon opening the app it’s immediately obvious the app isn’t design from the ground up for a VR experience. Instead it’s optimised for its primary audience of phone users.

On the home screen there’s nothing to indicate there’s a VR interface option at all. The search bar at the top of the page obviously requires keyboard input, the main map, and even the tiles at the bottom of the screen, all require touch and there’s no way of switching into a VR mode from this view.

Google Street View UI

A mobile first UI

Those tiles I mentioned, that’s the easiest way to get to the VR side of things. Scrolling down the page show a seemingly endless list of Google Street View locations that can be viewed using a cardboard setup.

Selecting a tile presents a street view style UI (still not in VR mode yet). This allows the user to tap the screen and move around the environment. You’d be forgiven for missing the small icon in the top right of the interface for switching to cardboard mode. It’s the one that looks like a little cardboard headset.

IMG_4296

Not exactly obvious how to get to VR mode

If you were lucky enough to make it this far, you’ll be presented with a standard Google UI telling you to turn your device on its side and put it into your viewer.

Google VR viewer UI

Standard Google viewer info screen

Once in Cardboard mode, you’ll experience a familiar interface to many other Cardboard VR apps. One difference here is they’ve mashed VR together with a desktop browser style Street View experience. Obviously to look around you just… well… look around 😉 but navigation is done via a point and click teleport interface. This uses the “on ground” style direction button similar to Street View’s desktop UI. It works well and it’s fairly intuitive given the navigation follows the users gaze. Basically where ever you look, if you can go there, the arrow will point the way. I prefer a more obvious waypoint style interface as it makes it clearer where you can go, but that’s just a personal preference.

Google street view VR mode

Explore Machu Picchu in VR

As I said at the beginning it’s obvious this isn’t VR centric app, but then again it’s clear that isn’t the intent either. Given the nature of Cardboard being a mobile based app there’s an awful lot of sense in adding VR as a feature, rather than the primary UI.

Although mobile users are the primary audience, Google still does a great UI job for VR users and that’s to be commended. Really we shouldn’t expect less from an industry heavy weight like Google.

Of course there’s always room for improvement, and this app is no exception. Street View being what it is doesn’t include any sound, it is just static 360 images after all. That being said it would be great to increase the emersion of the VR mode by including ambient environmental audio. It’s always amazing how good 3D sound can really put you in a place.

Conclusion

Ultimately Google Street VR did a good job of adding a VR mode feature to the mobile app. More could be done to improve the experience, but frankly this is an impressive example of 3D content captured for one interface and repurposing it for another. It’s not exactly a virtual world tour vacation, but it might just encourage you to take one.