The Reality Complex

Our future in Mixed Reality

Reality is an illusion, albeit, a very persistent one.

Albert Einstein

You’re probably reading this on your phone right now, or maybe a computer, or tablet, or perhaps some digital voice is reading it to you. Bottom line, you’re using a device that may have also told you today’s weather, showed you the latest headlines, made buying anything a click away and explained that your cough is probably some horrible disease. Perhaps that last one was a bit of misinformation but the point is, we now live in the technological information age. We are connected to everything and just about everyone through a multitude of devices promising a more convenient world.

While our human connections bloom, these same devices are shaping our realities with relentless, self-fulfilling persistence. Take ratings and reviews for instance. Placing our perception of the world on them may unintentionally obscure their objective truths when that objectivity can be visually altered. When constructing Mixed Reality (MR) environments we begin to merge our world with the limitless realities of our imagination and these truths regarding the world around us become more intertwined with the connected world.

What happens when the virtual world that exists on our phones, computers, and the internet at large is projected into the environment around us? All of those cat GIF’s, how-to videos, news reports, ads, Wikipedia entries, and customer reviews will become physical manifestations of themselves. Kinda sounds like a future dystopian film chock full of holograms, but our future will be far less obvious than most movies suggest.

When scientists and futurists discuss the future of human evolution, many conclude that a merger of mind and machine is inevitable. The result could be an ever connected culture, continually experiencing social #FOMO and a content industry clamoring for 100% of our attention. The Reality Complex, ranging from the real world to completely virtual environments is being navigated by more people on more devices as AR and VR become commonplace. When MR experiences become a more commonplace, the line between physical and virtual realities will become fundamentally blurred.

Our own biases will also play a large role in the designed realities we will inhabit. Implicit bias will influence our virtual worlds even more than they do our physical simply because they will be easier to change. The environment will be cloaked in a designed reality of one’s own making (or someone else’s). Will we be able to turn it off or see through it? Maybe the real question is; will we even want to?

Part 1

OUR CONNECTED CULTURE

Over the years we have grown accustomed to the conveniences the internet has brought us. That constant connection to people, places, and pretty much anything we might want to know, are at the touch of a button. So much so, that it has become hard to envision a world without this connection. Getting visual directions across town and reviews of a fancy floor mop all seem helpful enough. We’ve dictated our lives around this convenience without considering any of the negatives and why would we? We all want the best floor mop! However, as we’re beginning to see in the navigation sector, send one too many cars down the wrong street using algorithmic directions and the reality of our trust in given information becomes increasingly questionable. Cities are beginning to realize that even something as innocuous as algorithmic driving directions can be damaging.

While we continue to connect everything from lightbulbs to the kitchen sink, we are illuminating a fascinating truth in our species. Humans want to understand, contextualize, and stay connected to everything. While technology continues it’s progressive march forward we begin to look for familiar ways we can experience the world or acquire knowledge quicker. Augmented Reality (AR) allows us to manipulate and place content within a view relative to each of us. Virtual Reality (VR) provides our senses with an escape and designed point of view unlocking experiences only achievable in total immersion. When combining the two we get Mixed Reality (MR) where our real world and the virtual world combine to create something entirely new yet inherently familiar.


For as big a role this technology plays now, it will play an even larger one in our augmented future. We will create new technology to help us experience this world and our reliance on a constant connection will only become greater. Our use of social networks will evolve and largely define how we use this technology, much more than the individual promise of an interactive illusion.

What will our connected culture look like when we define ourselves from the mixed realities we create? Our society may grow and adapt based on the evolution of this world-augmenting technology. With this evolution comes new means of creating, interacting, understanding, solving problems, and experiencing the world around us. This is big. As big as hooking up a screen to a computer for the first time. In our future, we will see the world not for the way it is, but the way we want it to be. Our virtual space is merging with the physical environment and with it, a whole new medium is beginning to take shape. We will design for our homes, our running routes, parks, and offices. Our view of the world is changing. Ready to get creative?

Part 2

Creating for An Augmented World

Augmented Reality (AR) tech has introduced us to the very real future where virtual people, places, and things can exist in the world around us. Currently, the tech is limited to a handheld device such as your smartphone but the real magic kicks in when we make the leap from handheld to wearable. With it comes a shift in our perspective and creative approach. We make the leap to Mixed Reality (MR) when new technology can visually scan and understand your place in the physical world. The addition of computer vision technology and environmental data-collection enables virtual content to understand its own place in our physical world. Everything from a product installation at a store to emergency directions, will become spatially aware of our world.

A great example of a familiar MR experience would be maps. On your phone or tablet they show you roads, transit lines, traffic, and even local shops and ratings according to society (random people). This is a contextual layer that we’ve gotten used to consulting frequently. Before fancy phones or tablets, folding paper maps also provided this contextual layer over our world, though disconnected and far less convenient. With AR and MR technology we take this layer and place it out in front of us as we move about the world. All of that information will be visible and perceived the same as the practical objects that are physically there.

When we look through our screens into the world around us, the possibilities for MR content become endless. Stores will place MR ads right in their windows custom tailored just to you. Video feeds will become portals we will step into for an immersive 360 look all over the world. Even city parks will host art and events built entirely for a Mixed Reality audience. With our heads held high, we might just experience the world with fresh new eyes! There is a tradeoff here though. Much like the challenge of self driving cars, we need to be aware of distracted people failing to notice the real dangers around them. Elon Musk might need to create a self-walking human. With this virtual layer affixed with us at all times, we move about the world in a bubble of our own simulacrum.

Virtual content could mimic elements of our physical world or they could evolve into more fluid experiences. Narrative content can now exist in the world and create personalized moments for each and every viewer. For instance, an athletic company could design personalized running routes complete with a digital path or virtual running partner to follow, all projected out into the world in MR. Museums have already begun to explore AR content and contextual information paired with exhibits and famous pieces. Imagine an architectural tool that lets designers experience their future space then adjust a few walls by reaching out and pushing them. News feeds will deliver relevant articles based on your preferences, current location, time of day, and even, what you are looking at. Our future offices may transform into open tables devoid of screens, letting work content live in the MR view of the individual employee. The built environment will come to life as a sub-reality telemetric membrane is visualized all around you.

Location persistency will introduce an ability to place a virtual object in a location and have it remain there not just for you, but everyone else as well. This era of location persistent MR will inspire us to treat virtual content much like physical content. We may hold it, take it with us, and even have places around the home where we like to keep it. The persistent locations where that virtual content can exist may be controlled and sold much like a website domain or physical property. Just imagine: you own your house, the land it sits on, and the mixed reality space it shares.

We may be looking at a virtual land rush similar to the .com boom. DNS hosting services may offer GPS zones where buying the virtual GPS coordinates gives you ownership of the corresponding world area in MR. The question is; will we create content for a unified network much like the internet today or will every company strive to create their own version of this MR world?


Creating for this future is exciting and will require a mindful approach. Our future content will be cerebral, immersive, and designed entirely for us. The responsibility of creators is going to be in the content we create that lifts people up and out of their bubbles rather than cocooning them into a false reality of their own making.

Part 3

The Attention Economy

Attention is a commodity more valuable than any currency. The products you buy, content you consume, and places you go all feed a system designed to capture your attention and provide you with more stimuli. This is due in part by the growth of our connected culture as we increasingly take our lives online. In The Reality Complex, we’ve been exploring the social and creative considerations Augmented Reality, Virtual Reality, and Mixed Reality will bring to market. In Part 3 we look at what drives this technology: your attention.


We use smartphones a lot these days. An understatement to be sure, but pair that with our explosive growth in social networks and we come up with some pretty staggering figures. Consider that the average US user spends 50 minutes on Facebook alone in a given day, and that’s just one site* from two years ago. (Facebook 2016 Report) Another study found that young adults were spending, on average, five hours a day on their smartphones (PLOS ONE, Beyond Self Report). When we account for all time spent online to a connected culture, shopping, reading news, playing games, or even working, we begin to picture another world. A world shared and experienced in relative space. As we continue down this path, our relationship to our living reality will become intertwined with our relationship to our virtual one. In many ways, it already has.

We may invite it into our everyday lives where the dangers are disguised behind a promise of convenience. In a world where we share stupid cat GIF’s and Instagram celebrities secretly selling us fancy water, we are already giving our attention away. As we continue to live our lives online, corporations will benefit from our personal attention. They will create more ways for all of us to benefit off each other’s attention as well. Mindlessly binging on random video memes feels good to you and even better to the user getting paid for it. We’ve created a new economy driven by attention that everyone can participate in. The increased precision to which we will acquire attention analytics will mean more accurate targeted marketing where the goal is less about purchasing and more about retaining a consumer ecosystem. While we can’t guarantee that someone is actually watching the TV left on in the living room, we will know exactly what they are looking at in their MR headset at all times. It’s called “Retinal Tracking” and it’s going to revolutionize the marketing industry.


The value of Mixed Reality in our attention economy is the benefit technology brings to personal tracking. Learning more about you while you learn more about something else. In MR where content will be shared and exist all around us, the virtual and physical locations could become much like that of the popular Twitter feed or trending YouTube video: designed to capture attention for as long as possible. Even the physical locations in our “real world” will become trackable. The avocado stand at the grocery store is going to have an insane comments section. Best to ignore it.

Everything in our world can be considered content. Just as we do now with traditional ad media, we will track every facet of our MR future and strive to capture the attention of everyone as they go about their day. These moments of advertising will increasingly look more like recommendations or social discussions disguised behind the illusion of personal discovery. We will track how often people look at the fountain in the park just as precisely as we track a user’s buying habits.

“Why would anyone willingly participate in this?”

Consider this scenario: A product tech company introduces a new MR device and offers to lease it to anyone for free. As you use the device, it tracks you and the world around you, helping this tech company visually map and construct an image of the physical and digital world. Here’s the catch, if you want to use it for free you must leave the tracking analytics running at all times. If you were to disable tracking you will be billed $50 a month. In this scenario, everyone has the ability to participate regardless of wealth, though only the wealthiest can retain their personal privacy.


While this new personalized tracking may sound far fetched, we’re already inadvertently providing our data on just about everything we do. “Provide us with anonymous usage data to help make our products better”, “Based on your history you might like these other videos”, “Top 10 sandwiches near you”, “Can you ‘like’ my picture for me?” Our lives are tracked directly and indirectly every single day in the hopes to provide us with more relevant and personalized information. Creating this benefit is the tracking and categorizing of individuals as we move about the online world, and now, the physical.

It will be incredibly important to approach design with a mindful and ethically responsible focus as we create these connected, augmented experiences. We have a responsibility in our work now more than ever. We need to design experiences to be more inclusive, mindful, and empowering for everyone. Wielding the power of attention needn't be a concern. We should Consider every user in the experiences we create and every voice in the conversations we start.

We’ve reached a point where society has started to define itself by the technology we use. Our connected culture is driven by the attention we give it and is only limited by a screen that fits in our pockets. Augmented Reality has opened the door into a future where we can visually alter the world around us. That future comes with a connected society still seeking individuality and purpose. We take our first steps into immersive Mixed Reality and into a world re-imagined. Defined not only by the bricks and beams of the physical, but by the technological layers of human innovation and imagination. Our future will be of our own design, existing in a world where we will question every layer that makes up our growing reality complex.

MERGER

We strive to create a world perfectly tailored to how we want to experience it and Mixed Reality promises exactly that. A world that will let you see it however you wish. We champion the benefits of choice and rarely question at what cost. As we begin to introduce more and more Mixed Reality experiences into our daily lives, we will become more comfortable with this idea that something else can exist out there, in the world with us. Our reality is designed by our wants, needs, and actions. As we careen headfirst into this techno-revolution we must remember that as humans, it will be up to us to design that future. This comes with incredible responsibility to not only design a reality that looks and feels nice, but also design one that inspires, protects, educates, and elevates everyone.


This future holds the potential to be more inclusive if we’re able to pop our own biasy bubbles. Mixed Reality has that potential to elevate our everyday lives, but it also holds an inherent ability to alter reality. We must be mindful of the potential dangers this might introduce. It will affect all of us; our family, friends, the young, old, those with homes and those without. We must design for a better, more connected world while not losing sight of who we are and where we came from. Make the future work for everyone, not just ourselves. Make the future enjoyable for everyone. Our reality, after all, is not our own.

The Reality Project

2018