Hey, guess what? I’m learning iOS development and have been for months! Thought I might increase my rate of publishing to my blog if I did what I did for my VR course and posted study related stuff here. So to that end…
Here’s a simple step by step guide for adding CocoaPods to your iOS projects. There are quite a few steps involved and I suspect this is something that is done relatively infrequently in a single project. It’s always nice to have a reference for this sort of thing.
Inside Terminal, change directory to the folder containing the Xcode project
Initialise a new Podfile. You do this by using the “pod init” command. If you need more details read about it in the CocoaPod’s guide for pod init
Open the Podfile in Xcode. Just drag it to the Xcode icon in the dock
Add the library to the pod file. Here’s an example Podfile with SwiftyJSON and Alamofire libraries added
Install the pods using the “pod install” command in Terminal. Here are more details on pod install should you need them
Now open your Xcode project via the new .xcworkspace file
When I started my Udacity VR Nano degree I knew I was in for a challenge. Now that I’ve successfully completed my final assignment I thought I’d share some lessons learnt. So in no particular order:
Use version control. Thankfully I started doing this very early on. I had heard of the benefits and for whatever reason actually took them seriously. I started a religious habit of regularly pushing changes to GIT. I only used GIT to roll back to previous versions of projects hand handful of times, but had I not had the option I would have had a much bigger problem on my hands.
Don’t underestimate the time required. Taking on any study that requires large blocks of time can’t be taken lightly. It’s not going to be enough to only do a 30 min stretch at lunch, or an hour once the kids are in bed. You’re going to need a clear half day at least once a week to make real progress. Also, make sure you know what’s going to consume your time on projects in any given block. No sense clearing 4 hours for study only to have 3 hours of that time consumed rendering a 4k 360 video.
Don’t forget to look after yourself. With pressure on your time, it’s easy to cut out things like exercise or time with your family. Don’t do that. Cut out things that are actually a waste of time instead. TV is a big one but also think of ways of compressing your time. Like listening to audio books while you go for a run rather than reading a book alone.
Take breaks. If your mind is cluttered you’re not going to learn as well as you might. I found when I spent more than an hour trying to solve an issue the thing to do was take a 20-minute walk, clear my head, then take another crack at it. It’s amazing how often your mind will solve problems for you when you let it.
Take notes. Even if you never refer to them, take them anyway. Do this with a pen and paper (or a tablet and a pencil). It’s all about making the information get in your head. I almost never refer to my notes, I don’t take them to remember later. I take them to remember them now.
Try and integrate your life into your study. I did this by using my kids as the target audience for my projects and used them as user testing subjects while developing. It let me spend quality time with them and get some study done. It was also an education for them too so a real win-win.
Scope out what you’ll need before you begin. I started my course with a late 2009 Mac Mini. While old, it was fine for much of the course. The later parts that required 4k 360 video editing, not so much. This sort of thing can add considerable costs to your study budget so make sure you have a line of sight on this and save accordingly.
Get involved in the community. A huge part of successful study for me was the community of learners and educators around the subject. Help others and ask for help when you need it. Don’t waste time beating yourself to death if you get stuck. Solving your problem on a public forum helps everyone else with the same issue in future. So don’t be selfish, ask for help.
Be grateful. To your partner for looking after the kids. To your friends for listening to your ramblings about project issues. To your classmates and mentors for helping you along the way. Be grateful.
Stick it out. Software development is hard. It’s not supposed to be easy. It is literally constant problem-solving. There will be times you’ll think “maybe I’m just not good at this”. Just keep going.
I’ve been AFK for a while (holiday) so I just wanted to post the details of the final version of “night at the museum” I finished a few weeks ago that I delivered as part of my Udacity VR course.
The project went well (in that I passed) but I have to admit it was an exercise in restraint as much as anything. Like a lot of people, I tend to have big plans for everything I make but those plans are not always practical. This project was no exception. While I am ultimately happy with what I created, I do consider it very much a minimum viable project. When it comes to working on assignments like this there are a number of things to consider. First and foremost for me is available time.
I’m totally loving learning VR development, at this point, I spend almost all my free time doing it. But working full time and having a young family is a busy time in life, so if I want to progress projects I have to be efficient with my time or things can stagnate. Also, I currently pay for my studies via a monthly subscription so every extra month a project slips into comes at a significant extra cost.
Just to give you a few examples of compromises made on this project:
Playing content at each of the stations was fairly limited and doesn’t have much finesse. For example, the final build allows the user to play audio from all 5 stations at the same time, this leads to a fairly horrible experience if a user does this
I reused a museum model from a previous project because it was faster than building my own. This means it’s not really an ideal setup. Space feels a little constrained and doesn’t provide much room to move about. Most real museums are quite spacious so it didn’t really fit the aesthetic I was aiming for
I wasn’t that happy with the spatial audio implementation. The environment had a bit too much reverberated for my liking and given more time I would have improved this a bit
There’s no environmental audio, just the content of the stations. I’d preferred to add some atmosphere to the scene just to add the feel of the place
There are loads of other things that could be better about the final deliverable, but ultimately what was made “did the job”. Creating anything is always a series of compromises and I think delivering something you can live with and (resourcing permitting) can be built upon, is more important than getting things perfect. Perfection, after all, is the enemy of the shipped.
The more I create C# scripts in Unity the more I learn about optimising those scripts. With code there are 100s of ways to solve a problem, not all of them efficient, so learning to check your codes performance seems like a must.
Things to look for
So far I’ve come across a few simple things to look out for on the projects I’ve been coding. I’ll post more as I come across them in future.
Anything that’s called every frame creates consistent load on the CPU, GPU or both. If what’s called is inefficient or large it will have a negative effect on frame rate. In VR this is particular bad given low frame rates can make your user physically sick. Be sure to look at what’s in the Update and FixedUpdate methods and do your best to optimise the code as much as possible.
Working on mobile we will almost always be maxing out the graphics capability of the device. This effects the performance of the Update method. On the other hand FixedUpdated runs in sync with Unity’s physics engine. This will make sure your physics will be the most accurate and consistent. FixedUpdate is called every “physics step”, so is regularly used for adjusting physics (rigid body) objects. Make sure you always put your physics code in FixedUpdate.
Avoid using GameObject.Find. It basically requires the system to cycle through all of the GameObjects in the game to find the right one, which obviously is expensive for the CPU. If you know what object we are using ahead of time, define the object at the top of the script and reference it that way. GameObject.Find is useful for testing out ideas, but that’s about it.
As with most things in Unity, there’s a fairly reasoned logic to how this is done. You need three main things:
An audio source. You can think of this as the media player, it is literally the source of the audio within your environment
An audio clip. This is the actual audio file that will be played by your audio source. So think of this as the song that will be played by your media player
An audio listener. This is effectively your virtual ears within the virtual world. Normally this is attached to your main camera.
To give you an example of how I this works in practice. I first attached a GvrAudioListener to my main camera. This will allow my player to hear the sounds produced. Next, add a GvrAudioSource to a GameObject you want to produce a sound. Now load your audio source (your media player) with an audio clip. There are a few different supported formats, any old mp3 will work nicely. Just drag your MP3 into the audio click field of your GvrAudioSource. By default, the audio will play on wake, so just hit play to enter game mode and you should hear your new audio right away.
For more information on the ins and outs of Google Cardboard Audio capabilities, check out the documentation.
If you find your audio doesn’t play, try clicking edit -> project settings -> audio and make sure the “Spatializer plugin” is set to Gvr.
VR working properly again (at least I think I have)
select objects again
the FPS back over 60
Unfortunately I still can’t compile to my phone but hopefully, the weekend sorts that. It’s funny how breaking this thing has been possibly the most educational part of the course so far. I’m not saying I’d make a habit of it, but it was a net positive overall.
Making stuff is difficult, especially when that stuff is new. This is something I’m all too aware of. I was progressing nicely with my Night at the Museum project when (in my wisdom) I decided to upgrade my production tools mid-project. A word of advice, never do that.
Spending hours fixing a long list of issues is frustrating, but it isn’t a total loss. The key to success when you hit a bump in the road (like sucking at it, hating self etc) is to keep going! It’s not always easy but if you ever want to get good at anything, the key is preserving. Even in the face of self-loathing and a general distrust of your own abilities.
When you’re doing something new, embracing the tough times and roadblocks as an opportunity is key. You’ve got to see it as a way to toughen your resolve. If you can’t turn adversity into a win, then that adversity will win and you’ll stop progressing.
So the next time you find yourself making a virtual museum experience and your:
VR cameras stop working
you can no longer select items
your frame rate drops to a nausea-inducing 27 fps
you can no longer compile to a device
all of the above is your fault because you made an idiotic decision
delays cost you $300 per month in additional course fees (seriously send money now!)
Don’t give up, keep going! Not only that, tell other people about it. Seeing other peoples screw-ups is encouraging to others on the path to something new.
I’m working on a project for my studies with Udacity. The goal is to create a VR experience that demonstrates my research into a VR company. I’ll do a series of posts on this as I progress along the path to completion.
To start here’s my first crack at the initial documentation for the project:
Statement of purpose
The purpose of the "Night at the museum" application is educate the user on the HTC Vive VR solution. The experience is based on an exhibit style scene, which contains 5 "stations", each covering some aspect of the HTC Vive.
The target audience for this application is anyone interested in VR that would like to learn more about the HTC Vive VR solution. Here’s a basic persona from them:
Name: Wade Watts
Role: Budding VR enthusiast
VR Experience: Experience with google cardboard but limited beyond that
Quote: "VR is very exciting, I can't wait to learn more"
About this person: Wade has grown up with computers, game consoles, tablets and smart phones. He's a digital native that loves to escape into his devices. VR is the ultimate expression of that escape.
I’ve started by looking at different museum and exhibit styles to select a something that fits the aesthetic I’m looking for. After looking at altogether too many museums (if that’s possible) I eventually settled on classic Greek style building with exhibits setup like sculptures on pedestals. I’m using the following images as inspiration for the design:
I like this style, it reminds me of some of my favourite museum tours, and I’ve also done some work in a previous project that shares a similar style. This should allow me to reuse some of the assets from that project, meaning I can move faster. Always important when you’re pressed for time.
Having established a concept for the style of the experience a setup to make a concept build. Using the assets from previous work made putting something workable together relatively easy.
I added a terrain GameObject with a few mountains in the distance to give the scene a sense of depth. I dropped in a “temple” model that is a good fit for a museum and added a few items to flesh out the concept. For the actual stations I acquired an HTC Vive model from sketchfab user Eternal Realm. I also kept an eye out for the scenes frame rate to make sure I wasn’t introducing anything at this early stage that had a big impact on performance. I still have a few more models to acquire but I’m happy with the direction the concept is heading in.
Now that I’ve got the basics in place the next thing on the agenda is the first round of user testing. I shouldn’t have to much trouble acquiring a suitable test subject to get things moving. I will be interesting to see how the scene changes and progresses over the course of development.
As discussed earlier, there are loads of terms floating around in the world of VR. But it’s not just things on the spectrum between AR and VR. There are also various types of experiences within just VR.
Rather than try to cover every possible variant here, I’ll talk to the types I’ve experienced personally, and as a result what I think are the important components that make for good VR.
I’ve used quite a range now, from 3 degrees of freedom (3DOF) Google Cardboard, right up to a high immersion 6 degrees of freedom (6DOF) HTC Vive setups. Here’s an overview of what I’ve used so far:
I’ve got what can only be described as a growing collection of Google Cardboard headsets. The more you get involved/exposed to the VR community and events it seems sort of inevitable that you’ll pick up a few along the way. This is part of what makes Cardboard so great. It’s highly accessible. If you’ve got a modern iOS or Android-based phone you’ve got 99% of what you need to get going. Cardboard headsets range in sophistication and price, from $15 bits of literal cardboard, to “fancier” setups that strap to your head with headphones for $50+.
Accessibility aside, the sorts of experiences you can do in a Google Cardboard really are a perfect entry into VR. It’s a fantastic way to take users to inaccessible locations and show them things. Examples include estate agents showing an out of town client through a property, or placing a marine biology student at the bottom of the sea. The biggest and most obvious shortcoming is the low levels of both emersion and interactivity available to carboard users. Effectively you can only interact via the users’ gaze or a single button interface on the headset itself. This sort of thing is fine for a certain segment of VR but obviously, you’re not exactly going to be entering the Matrix anytime soon.
One of the things that can’t go unsaid about Cardboard’s accessibility is how it lowers the barrier to entry to VR development. As I said in my hardware for VR post, you can get into VR on a very low budget with cardboard as a target device, and that’s a big deal.
The Gear VR is a fascinating step up from Cardboard. In many ways, it’s a very sensible direction to take VR. At around $100 USD, plus the cost of a compatible Samsung phone, the cost is more accessible than a full emersion setup. Obviously, a phone strapped to your head is only going to take you so far visually. You’re also only going to get a 3DOF (that’s pitch, roll, and yaw tracking of the users head) which limits the feeling of presence. However, the addition of a controller, even a simple 3DOF one, means far greater levels of interactivity. To my mind, having any control scheme that simulates a users hand greatly increases the value of the setup.
Having said all that, the Gear VR isn’t something I’d recommend unless you’re already a Samsung phone user. Its benefits are not worth the $1000 NZD investment over a cardboard setup. It’s also worth mentioning this segment of the marketing is increasingly competitive. Google have Daydream in this space. I haven’t used it personally but conceptually it looks quite similar. If you think a setup like this is something you’d like to explore you should consider both before making your decision. You should also be aware that Daydream looks to be releasing stand-alone headsets later this year that may be the most interesting take on this segment yet. Unless you’re in a mad rush for a VR setup I’d consider waiting until later in the year before buying.
HTC Vive (6DOF + 6DOF hand controllers)
I like to think of the HTC Vive as a window to the future rather than the revolution that some tout it to be. The experience itself is unmistakably amazing. The level of immersion is so high that I stopped thinking about controlling a computer simulation, and started just naturally interacting with it as if it was a real environment. It’s actually quite a strange thing to describe, but having used this setup dozens of times I’m convinced this as far more to do with your hands than it does your head.
The second you enter a VR environment, and you have some control of it via controls that closely approximates your hands, you stop thinking about the tools and just start being. Before you know it you are truly in VR. Based on the impact of hand controllers I’m a bit surprised more isn’t being done with cardboard headset combine with hand controllers, but then again there are likely significant technical challenges I’m not aware of.
Unsurprisingly the Vive is by far my favourite VR experience. However, the reason I say it’s a window to the future is the ticket price. These things cost a fortune. A Vive with hand controllers cost $799 USD. Which on its own is a big investment for most. Not inaccessible, but significant. The real kicker here is you need a high-end Mac or PC to connect the headset too. Taking that into account you can be looking at a $3000+ NZD machine to get started. Not exactly mainstream pricing.
It’s just a matter of time before these things transform into a more mainstream format. It’s quite likely the coming stand-alone headsets are the beginning of that. How much they cost and how good they are is yet to be seen, but I for one am excited at the possibilities.
The sooner more people have immersive VR experiences and reasonable consumer prices the faster this industry prices. With gateway drugs like Cardboard giving people their first taste for a low price, it’s sure to an explosive industry once the hardware catches up.