What Mixed Reality Actually Does

Most discussion of XR — augmented, virtual, mixed — focuses on the technology. Headsets, tracking, rendering quality, latency. The harder question for designers is what these technologies actually do to perception, and how to use that effect on purpose rather than by accident.

This is a piece about a specific finding from our practice: mixed reality's strongest effect is not immersion. It is the transition between real and virtual — and that transition, designed deliberately, deepens the impact of an experience rather than diluting it.

The argument rests on the four soft criteria — Relevance, Immediacy, Efficiency, Elegance — that we use to judge mediatecture designs. Those are introduced in a separate piece in this section. Here we apply them to a single question: what does mixed reality let us do that other media cannot?

We first noticed the effect in our own building. At the time, our Berlin studio occupied a decommissioned funeral hall — today we occupy a decommissioned church, but the relevant building here is the Luxoom Lab as it then was: a single brick-walled room with strong architectural character. When we were asked to open the studio for a public design week, we built a VR showcase that linked several of our projects into a single guided tour.

We needed a lobby for the tour — a space where visitors unfamiliar with VR could orient themselves before navigating into the projects. Rather than build a generic virtual lobby, we replicated our actual hall in VR. In the physical room, we projected a large portal onto one wall. In VR, the same portal appeared in the same position, leading into the projects. The visitor put on the headset standing in the real hall, and the first thing they saw was the same hall, with the same portal, now functional as an entry point.

The staff supervising the showcase noticed something unexpected. Visitors did not take off the headsets when they finished. They navigated back through the portal, returned to the virtual replica of the lobby, and only then removed the headset to find themselves standing in the real version of the same space. They had treated the virtual lobby as the place to land before leaving, even though removing the headset at any point would have returned them to physical reality.

Something had bonded the two spaces in their perception more tightly than the technology, on paper, should allow.

When we wrote Handbuch Mediatektur, we drew on a constructivist account of perception to explain effects of this kind. The short version: what humans perceive is not a direct readout of the world but a construction the brain assembles from sensory input and prior expectation. The brain treats this construction as stable and reliable, and most of the time it is.[^1]

Cross-referencing between senses is one of the brain's main tools for deciding what is real. A brick wall is reliably solid because vision, touch, and prior experience all agree. In a virtual environment, those cross-references are partly broken — but they are still active in the background, looking for matches.

What happened in our lab was that those cross-references found matches where they should not have. The physical room and its virtual replica shared the same proportions, the same brick walls, the same portal in the same position. The brain registered a match, treated the two as continuous, and stopped flagging the transition between them as a category boundary. Removing the headset abruptly broke that continuity — so the brain looked for a way to return to it before exiting.

The effect required no special technology. It required only that the real and virtual environments share enough perceptual anchors for the brain to treat them as one continuous space.

This is a different proposition from how augmented and mixed reality are usually framed. Augmented reality is typically described as adding information to reality — a layer on top of the world. Useful, but additive.

What we did in the lab is closer to what the term mixed reality can mean when it is taken seriously: a transition between real and virtual in which elements of the environment stay continuous across the boundary. The real and the virtual are not stacked. They are stitched together at carefully chosen points.

When the stitching holds, the experience is no longer two experiences glued at the edges. It is one experience that happens to use two media to deliver itself. That is what makes the transitions worth designing rather than minimising.

We pitched the same idea to CLAAS, the agricultural machinery manufacturer. The product was a combine harvester — a machine whose engineering strengths are entirely hidden inside its structure. From the outside, one combine looks much like another. What distinguishes a CLAAS combine is what happens inside it: how the threshing mechanism handles the crop, how the flow of material is managed through the body of the machine, how the engineering choices translate into yield and reliability in the field. None of that is visible when the machine is parked at a trade fair.

Rather than use VR headsets, we used an iPad as a tracked camera connected to a rendering rig. The iPad served as both viewing device and tracking reference. The screen — at larger venues, a substantial LED wall — showed what the iPad's virtual counterpart was looking at. The whole setup was mobile, scaled to audiences from small product workshops to full trade fair stages, and operated by a CLAAS product expert who used the iPad in hand to walk visitors through the machine's features.

The sequence opened with the virtual camera somewhere out in a wheat field, watching the combine at work mid-harvest. The camera then turned and approached the machine, closing distance, until its angle and position matched the iPad's view of the physical combine standing on stage — a stationary, non-operating machine. The transition resolved cleanly: same machine, same perspective, same scale, suddenly real.

From there, the show moved between modes. The product expert could pull the threshing mechanism out of the machine and let visitors see it operating in isolation, the grain flowing through. They could show the path of the crop from the cutter bar through to the grain tank, with the body of the machine made transparent for the duration. They could return to the field view to show the combine in context, then come back to the physical machine on stage. Real and virtual, operation and stillness, digital and steel — every transition designed to align rather than to call attention to the boundary.

Talking to the audience afterwards, the test held. Visitors did not separate the real machine from its virtual counterpart in their descriptions. They talked about what the machine did. The mixed-reality content had been absorbed into their perception of the physical object on stage. I used to put it this way at the time: if we had shown gummy bears flowing through the threshing mechanism instead of wheat, the audience would have come away believing they had seen gummy bears moving through the actual machine. The medium had stopped being readable as a separate medium.

The four criteria gave us a way to read the result.

Immediacy was structural to the approach. Where conventional XR creates immersion by replacing reality, mixed reality of this kind creates immediacy by keeping reality intact and weaving the virtual through it. The visitor does not have to translate between a real thing and a separate representation. The two are one.

Relevance came from what the virtual layer revealed. The internal workings of the machine — the source of its technical advantage — were the actual subject of the communication. The virtual layer was not decoration. It carried the message that the physical object on its own could not.

Elegance sat in the choice of device. An iPad is a familiar object. People understand without instruction what it is and how to look through it. The technological threshold for the audience was effectively zero, even though the rig behind it was complex.

Efficiency followed from the same choice. No headset distribution, no individual VR sessions, no specialised hardware in front of every visitor. One operator with one iPad could address a full audience, with the screen serving as the shared view. The technology stayed out of the way of the experience.

The approach scales beyond machinery on a stage. Anywhere a brief involves a physical object whose significance lies in something invisible — inside it, behind it, before or after the moment of viewing — mixed reality of this kind has structural advantages over either pure VR or conventional AR. The continuity with the real anchors the experience. The transitions carry the message.

The CLAAS work was developed under the project name SHOW SMART and later productised into a system called REVELATE XR. The product exists, has been used on selected projects, and remains available — though not as a high-volume offering. The more durable contribution is the design principle: that the value of mixed reality is in the seams, and that the seams are something to design carefully rather than hide.

The four criteria, applied to XR, point to the same conclusion they point to in any medium. The technology is not the deliverable. The control over what the technology does to perception is.

---

Originally published 2022 on aixr.org. Edited.

[^1]: For the constructivist account of perception, see Peter Kruse and Michael Stadler, "Der physische Apparat des Menschen," in K. Merten, S. Schmidt, S. Weischenberg (eds.), Die Wirklichkeit der Medien, Westdeutscher Verlag, 1994; and Peter Kruse, "Stabilität – Instabilität – Multistabilität. Selbstorganisation und Selbstreferentialität in kognitiven Systemen," in DELFIN XI, Iss. 3 in year 6, 1988.

Tobias Sievers · 2 April 2026