The Tomato-Throw Game, By Microsoft HoloLens and Oshyn

At Oshyn, we like to stay ahead of the rest in terms of technology. One of the things we know is “next” is mixed reality. We believe this technology will become as pervasive as our cell phones are today, but it will take perhaps 10-15 years to get to that point. In the meantime, however, there are still some pretty neat things we can do with the technology and, to that end, we've embraced what we believe will be the next big Microsoft platform: Microsoft Holographic aka Microsoft HoloLens.

What is Microsoft HoloLens?

Everybody understands what virtual reality is – a fully immersive environment that a technology places you in. Think Facebook's Oculus Rift. HoloLens is similar in that it is a headset that projects 3-D objects into your view. The key difference from a virtual reality platform is that HoloLens merges these 3-D objects into the current world you are in – it “augments” and actually “mixes-in with” your reality. You no longer are required to leave your world. HoloLens instead merges holograms into your world. This sounds like a simple change at first, but the technology required to make this happen is vast and complex. At a basic level, the HoloLens scans your surroundings using a series of sensors and cameras, tracks your exact head and arm movements, adjusting and re-rendering holograms at 60 frames per second, and partially or fully blocking-out items (this is called occlusion) that are behind real-world objects in order to create a more believable reality that fully blends-in with your world. The fact that you don't leave your world while using HoloLens is significant because it means that the augmented experience does not tire you out, allowing you to eventually become accustomed to using it 24/7 as the headsets become lighter and less intrusive in the future.

If this is your first time reading about Microsoft HoloLens, we recommend you look at the following links for some of the basics:

Microsoft HoloLens Website

Microsoft HoloLens YouTube Channel

What is The Election Tomato Throw?

At Oshyn we truly believe in having fun while we work. When work isn't client facing, we often take this concept to a whole new level. To that end, we made a fun little game for HoloLens users to try on their new devices. We made this mostly as a proof of concept and to start getting our staff trained in both design and development for HoloLens. We all know that the 2016 presidential election is a little different than most in that both major party presidential candidates are essentially disliked by a large segment of the population. Therefore, we thought it might be entertaining to make a little game where you can throw tomatoes at them. Inside a holographic game environment, of course. In the game, you take your pick – either Donald Trump or Hillary Clinton – and you use a slingshot to pull back, aim, and throw a tomato projectile that, with some practice, can hit the presidential candidate dead-on. See our trailer for the game below.

Lessons Learned

We learned a number of lessons from this experience in relation to Microsoft HoloLens.

• Design is a whole new ball game.

  You of course need to experience using a HoloLens in order to design for it. Experiencing one allows you to realize that you will, for example, need to make buttons big so that they are easy to target. It doesn't quite end at simple things like this, however, there are many nuances that need to be thought through and prototyped in order to make the experience feel natural to users who most likely don't have a lot of experience using a Microsoft HoloLens. You should therefore save plenty of time for trial and error and you should prototype interactions early and budget for things to be redone a few times before interactions feel right.

• Gesture recognition is still pretty rudimentary and limited.

  Most people don't intuitively know how to “tap” the HoloLens way and, even when they do, they want to interact with objects the way they do in the real world. The current APIs don't really support flexible gestures, as essentially there's only one gesture that HoloLens apps must use to begin any interaction. For this gesture, you make an L with your thumb and index finger right in front of your face, at least a foot or so away from your hand, as you point up. After this, the device can track your hand as it moves, but users often want to move their hands as they would in the real world beyond the range that device is currently able to track hand movements. You should therefore, as noted in the prior bullet, test-out your interactions early and budget for things to be redone a few times before interactions feel right.

• 3-D Sound is amazing.

  A huge and unexpected plus is the quality of the sound experience. 3-D spatialization of the sound, as well as the fact that the HoloLens speakers do not cover the ears, does wonders for making people feel the experience as real. By not covering your ears, sounds from the HoloLens blend with sounds around you. Furthermore, just as the holograms move as your head moves, so too does the sound. Users of our game truly feel like they were at a Trump or Clinton rally when playing them game.

• Voice recognition is reliable.

  I would even say it's great, other than the fact that everyone seems to have this right as well. The API for integrating voice recognition is also pretty simple in terms of learning curve and development effort required to integrate.

• You need developers with a solid math background.

3-D Vectors and Quaternions across multiple axes are computations developers will need to code often. This math can often be a little mind boggling and having a solid linear algebra background is highly recommended, if not required.

• APIs have a long way to go

This is not so much a criticism as simply a statement that this entire platform is completely new. It's as if we had canvas on which to draw APIs that allows us to draw lines and shapes in our screens. However, the higher level user interfaces, interactions and gestures are still rudimentary as we don't yet interact with our devices like this. This is a huge opportunity for people to do for holographic what Apple did for cell phones when it developed the touch interface. Right now, the defining method for how we will interact with Holograms is yet to be devised.  Once that nut is cracked, the APIs for all platforms will fall in line.

Who Will Win the Election?

The results from our HoloLens game seem to speak for themselves and were tracked live. As of the time of this writing, results show that this will be a landslide “victory” for Donald Trump as far as having gotten the most tomatoes thrown at him within the game, though Hillary Clinton got her fair share of tomatoes as well.

Have a HoloLens Application You'd Like to Pilot?

If you are an enterprise customer considering a commercial application that uses HoloLens, you've come to the right place. As Wave 1 HoloLens developers, we have been piloting concepts for a few months now and we would love to discuss a pilot or prototype with you. If interested, feel free to fill-out the brief form below, and we will be in touch with you shortly to discuss your needs.