The release of Apple’s ARKit and Google’s ARCore has sparked a burst of creative energy and a wave of demos from developers all over the world. And there is a growing possibility that a paradigm shift in computing could be just around the corner when AR takes over mobile as the dominant consumer platform.
As a veteran in the AR space, we’ve been waiting for this moment for a long time, and 100,000 hours later I can tell you we’ve learned the ‘harsh reality’ part of augmented reality first-hand. There is a long road between demo and live project and materializing any project into reality (or augmented reality) can take much longer than you might expect. These early demos we’re seeing will still run into similar marketing challenges we faced in the early days of building AR experiences.
Sometimes the challenges should have been obvious, but more often than not they come from the messy process of invention and applying innovation to the real world. The results are worth it. Consumer engagement with AR provides companies with tremendous opportunities to improve products and tell brand stories.
Here are our top considerations from the frontlines of AR/MR that developers should take into account when building mixed reality experiences.
1. Hardware limitations
Both ARKit and ARCore target later release phones. So a large number of consumers will be left out until they upgrade their hardware. Neither Google nor Apple’s solutions are currently cross platform, so developers will have to port from Apple to Android or visa versa to accommodate the most users possible.
2 – You still need an app
ARKit and ARCore are both integrated into the OS of the device which provides great performance advantages over third party software which sits a layer above. However, the code containing the content and the experience still requires an App download. Creating the app is just the tip of the iceberg, marketing the app and driving downloads are the most daunting challenges(of many) that lie beneath the waterline.
3 – Nobody wants a one trick pony
To make an experience worth downloading, developers should avoid any singular experiences. You can have the coolest animation in the world, but it will grow tiresome after a single repeated viewing. A good question to ask is what can the user do the first second, the first minute, the first day and the first week, and make sure the experience evolves. The only way to solve this besides widening or deepening the experience is coming up with a dynamic solution that’s different every time. The Machines is a good example of using multiplayer to make a game replayable even if the content runs out.
4 – 3D is expensive
Budget is often the “dream killer” and much of that can go into creating 3D assets. Clients either don’t have 3D assets readily available or if available they are not in a usable format. There can be a lot of work involved bringing an asset from film/VFX resolution or from CAD manufacturing files, down to what will work on a consumer’s mobile phone. The amount of 3D required is also an issue. As mentioned before, variety of content is important. Sometimes though it can be an issue of “completeness”. For instance, it’s easy to demo placement of a single coffee table, but what happens when the brand has hundreds of products? The IKEA Place app is a great example of a large catalogue of 3D objects with over 2000 items available in AR!
5 – Locations can be constraining
There’s a reason why so many ARKit demos are shown in wide open areas. Collision and occlusion are a huge issue and ARKit and ARCore can’t detect or account for this. So that same demo that looked great in a wide open space, suddenly looks very broken inside a coffee shop or in a class room. There are many more considerations when location comes into play. We’ve had to work around fast food table sizes, ideal foot traffic in retailers, changing lighting conditions in theme parks and even magnetic interference around landmarks to name a few.
6 – Avoiding Fatigue
Holding up a phone, which we call the “device up” experience can be surprisingly tiring. And years of prototyping and user-testing have taught us that different user types have different conditions or thresholds for fatigue. Young kids for example have trouble holding up a phone and tapping the screen, or may not even be able to steady an iPad with two hands. While an adult out shopping will only have one free hand for their phone. Fatigue also affects hardware; AR requires an open camera which drains the battery very quickly and heats up the device.
7 – Being Social
AR is most powerful when paired with other technology. User’s naturally want to share their AR experience because it’s the only technology that places 3D content into THEIR world. In addition to selfies, Snapchat’s world lens uses AR to create sharable scenes. But social is only the start. Live data can make AR timely. GPS and beacons can make it location aware. And a variety of enterprise features can make AR practical and useful.
8 – Consumer Messaging
This is probably the most neglected and difficult part of the equation. A consumer wants to know why an app is worth downloading and feel comfortable with how it works before they commit. This communication can happen across different media from TV, to digital, to print. Or it can end up falling on the shoulders of an employee in store. Once a user downloads the app, they must be instructed on how to use it and what to do if things appear not to work.
9 – Don’t forget the Physical Part of Digital Physical
AR in an open space or on an empty table can be a very engaging experience. But we’ve found that AR becomes exponentially more magical when it involves something physical under the digital layer. Placing a digital character is fun, but it’s even more fun when you can walk that character up to a toy car, open the door and then drive off in it. Or a virtual pet is cute running around on the grass, but it becomes much more interesting when it looks for a real tennis ball, or knows to scratch at the door to go inside. In the Chalk AR app you can draw over the real world in AR with another person. That layer of interactivity with the physical world can make all the difference in an AR App.
10 – This is just the beginning
Last year Pokemon Go introduced the world to AR. This year ARKit and ARCore will deliver the first true AR experience to many consumers. While existing AR platforms, like Vuforia, are already integrating with ARCore and ARKit to enhance its features. These technologies are just scratching the surface and the challenge for developers will be that there are so many more capabilities just around the corner. Just when we think we’ve sorted out mobile AR, head-mounted AR will finally mature and shift the whole industry once again.
Jason Yim is the CEO and executive creative director of Trigger, the largest mixed reality agency with more than 90,000 hours of development in AR/VR/XR.