Extended Reality (XR)
Put the information where the work happens.
Extended reality covers augmented, mixed and virtual reality: three ways of putting digital information into a person's field of view. ColdAI treats the choice between them as an engineering decision driven by the task, the environment and the data, not by a device catalog. We start from what the worker has to see and do, compare XR honestly with a tablet or better documentation, and build on open runtimes so the application can outlive any single headset.
The opportunity
Start with a decision
worth improving.
A field technician faces an unfamiliar repair on site. A headset overlays the relevant procedure on the equipment and connects a remote expert who can see what the technician sees.
From possibility to a working system
Follow the flow.
Three connected decisions. One considered architecture.
Conceptual flow, not a live system or measured result.
Choose the right reality
Decide between augmented, mixed and fully virtual experiences based on the task, environment and safety constraints. Some workflows are better served by a tablet, and that comparison belongs in the plan.
Anchor the content
Align 3D models, procedures and live data to physical space using spatial anchors and the asset's digital record. Build on open runtimes such as OpenXR or WebXR to limit device lock-in.
Measure in the field
Pilot with real users in the real environment. Track task time, errors, comfort and accessibility, and plan fleet management, updates and the handling of camera and eye-tracking data.
The work, made concrete
What we can build
with your team.
We agree scope, dependencies and acceptance criteria before delivery. Your project can start with an assessment, a pilot or an integration into an existing system.
XR use-case and device assessment
Clarify the system boundary, the responsible owners and the decisions the architecture needs to support.
A spatially anchored pilot application
Turn the agreed design into reviewable work, evaluated against representative inputs and explicit success criteria.
Field evaluation and fleet operations plan
Make the next step operable: document responsibilities, known limitations and the path from pilot to ongoing use.
In depth
Go deeper: platform and fleet decisions
Two decisions apply whichever modality you choose: the runtime your application is built on, and how the devices are run once they leave the box.
- 01ComparisonOpenXR vs WebXR vs native SDKs: choosing an XR stackCompare OpenXR through an engine, browser-based WebXR and native platform SDKs on device reach, feature access, distribution and portability, then pick a route.
- 02ProcessEnterprise XR headset fleet management and securityA seven-step operating process for enterprise XR headsets: manageable devices, enrollment, kiosk mode, update rings, shared use, sensor privacy and retirement.
Matching a spatial task to AR, MR, VR or no headset at all
Start from the work, not the device. The right answer for a particular task can still change once people try it on site.
- If
The worker must keep their eyes on real equipment and both hands are busy, for example during a repair or an inspection.
ThenUse AR on a see-through headset, or mixed reality on a passthrough headset where the site's safety rules allow it.
Instructions and readings appear on the object itself, so the worker does not look away to a manual or a screen.
- If
The task happens occasionally, many people need it, and issuing headsets to all of them would be hard to justify.
ThenUse AR on the phones and tablets people already carry, as described on the augmented reality page.
Distribution and support are far simpler, and hands-free operation matters less for short, infrequent jobs.
- If
The task is dangerous, expensive or impossible to practice on live equipment, or the space has not been built yet.
ThenUse VR for rehearsal or full-scale review; the virtual reality practice covers scenario design.
Full immersion replaces the environment instead of overlaying it, which is exactly what you want when the real one is unavailable.
- If
Remote specialists need to see what a field worker sees and mark up the scene in front of them.
ThenBegin with remote assistance on existing phones, then move frequent users to headsets once usage justifies it.
The value lies in the shared view and the annotation, which a phone can deliver before any headset fleet exists.
- If
The information is not spatial: a policy, a form or a sequence that does not depend on where things are.
ThenDo not use XR. Improve the documentation, the video or the software instead.
A headset adds cost, comfort issues and device upkeep without giving the task anything it needs.
How AR, MR and VR differ on the constraints that decide projects
| Constraint | Augmented reality | Mixed reality | Virtual reality |
|---|---|---|---|
| What the user sees | The real world, with digital content on a see-through display or a phone camera view | A camera feed of the room (passthrough) blended with digital objects that respond to its surfaces | A wholly digital environment; the real room is hidden |
| Awareness of surroundings | Highest: people see their surroundings directly or around a hand-held screen | Good, but limited by passthrough quality, delay and field of view | None; needs a cleared play area or a seated set-up |
| Typical devices | Phones, tablets and see-through headsets | Standalone headsets with color passthrough cameras | Standalone or PC-tethered headsets |
| Alignment work | Precise registration to specific objects with image targets, object tracking or anchors | Room understanding through planes, meshes and anchors so content sits on real surfaces | Little or none; the whole scene is authored |
| Content effort | Procedures and data overlays tied to real assets | 3D models that must look plausible beside physical objects | Complete environments built to the fidelity the objective needs |
| Typical fit | Guided procedures, inspection rounds, expert annotation | Layout planning on the shop floor, collaboration around physical products | Rehearsing hazardous or rare tasks, reviewing spaces before construction |
Vendors use these labels loosely, and many headsets run more than one mode; confirm what the specific device supports.
Where the data behind an XR application lives
Most of the engineering in a useful XR application sits below the headset. Content has to come from the systems that already own it, or it goes stale the first time a procedure changes.
- Headset or device app
Renders content, takes hand, eye, voice or controller input and reports what the user did.
- Spatial alignment
Anchors, image targets or object tracking that tie content to a place in the room or a point on an asset.
- Content and scene services
Serve 3D assets, procedures and annotations, versioned and cached for weak networks.
- Integration APIs
Scoped read and write access to business systems, limited to what each user may see and change.
- Systems of record
Maintenance, asset, product lifecycle, learning and document systems that stay the source of truth.
Keeping XR content tied to live operational data
ColdAI's XR work connects devices to the systems that hold the underlying data rather than copying that data into the application1.
The quickest way to build an XR demo is to bake everything into the app: models, steps and labels. The quickest way to retire it is the first procedure revision, which then needs a new build, a new test cycle and a new push to every device. In production, the app should fetch procedures, asset attributes and work orders from the systems that hold them, so a change made by engineering or maintenance appears in the headset without a release.
Asset identity comes first: each anchored item of content needs a stable link to an asset record, not to a position in a scene file. Models usually start as CAD or BIM data far too heavy for a standalone headset, so a conversion pipeline simplifies geometry and exports it to a runtime format such as glTF; the spatial computing practice covers that pipeline and persistent anchors in depth. Writes need the same care as reads. A reading captured in AR should land in the maintenance system with the user's identity and a timestamp, under the same permissions as the desktop screen.
Connectivity is the constraint teams underestimate. Plant rooms, vessels and substations often have weak coverage, so the content service has to cache what a job needs before the worker walks in and queue writes until the device reconnects. Building on open runtimes such as OpenXR keeps the device layer replaceable2; the data layer should be just as independent, so switching headsets never means rebuilding integrations. The runtime comparison explains how to choose that device layer.
What a field pilot should measure
A pilot needs a baseline and measures that operations leaders already trust. Record these for the same tasks done with and without XR.
Why XR programs stall between pilot and rollout
Novelty inflates early results
Early signalEnthusiastic feedback in the first sessions with no measured change in task time or errors.
MitigationRun the pilot long enough for the novelty to fade, and compare against a group doing the same tasks the usual way.
Nobody owns content upkeep
Early signalProcedures shown in the headset lag behind the official versions.
MitigationFeed content from systems of record and give one team ownership of the XR content pipeline before rollout begins.
Dependence on one vendor's business program
Early signalThe management console, app store and business device line are all controlled by a single supplier.
MitigationBuild on open standards where possible, keep assets in neutral formats and watch vendor notices. For example, Meta stopped selling Horizon managed services and commercial Quest models from February 20, 2026, while committing support for existing customers until January 4, 20303.
Sensor data treated like ordinary telemetry
Early signalCamera feeds, room scans or eye-tracking data are stored by default with no stated purpose.
MitigationClassify sensor data before the pilot, switch off what the task does not need and set retention limits; the headset fleet guide sets out the controls.
People who cannot use a headset are left out
Early signalDrop-outs cluster among people who wear glasses, are sensitive to motion or have limited mobility.
MitigationDesign an equivalent non-headset route from the start and test fit and comfort with a representative group of users.
Frequently asked questions
Is mixed reality the same thing as augmented reality?
They overlap, and vendors use the terms differently. In practice, augmented reality usually means digital content laid over a direct view of the world, on a see-through display or a phone screen. Mixed reality usually means a headset that films the room and blends digital objects into that feed, so they can sit on tables or be hidden behind real objects. What matters for a project is whether the device shows the surroundings well enough for the task and the site's safety rules.
When is a tablet a better choice than a headset?
A tablet is often better when the task is occasional, when many people need access, when work happens outdoors in bright light, or when the information is glanced at rather than followed step by step with both hands. Tablets are cheaper to issue, easier to share and already supported by IT. Headsets make sense when hands-free work, accurate placement on equipment or full immersion is central to the job and people use them often.
Do we need 3D models of our equipment before starting an XR project?
Not always. Many AR procedures work with photos, image targets or simple markers, and remote assistance needs no models at all. Where models help, existing CAD or BIM files are the usual starting point, though they need simplifying before a standalone headset can render them smoothly. If no models exist, a pilot can start with the few assets that matter most, modeled only to the detail the task requires.
Can an XR application work without a network connection?
Yes, if it is designed for it. The application downloads the procedures, models and asset data a job needs while connected, runs from that cache on site and queues readings or sign-offs until it reconnects. The harder questions are freshness and conflicts: how old cached content may be before it is blocked, and what happens when two people update the same record offline. Agree those rules with the system owners.
How does ColdAI decide whether an XR project is worth doing?
We start with a specific task and compare the XR option against a credible conventional baseline, such as better documentation, a tablet application or video guidance, as our frontier technologies practice does for every emerging technology. If the conventional option solves the problem, we say so. Where XR does look worthwhile, the first deliverable is a use-case and device assessment, followed by a pilot measured in the real environment with real users.
Sources
- Extended Reality (XR): ColdAI's approach to AR, MR and VR applications — ColdAI
- OpenXR: high-performance access to AR and VR — The Khronos Group
- An update on Meta for Work — Meta
Connect the architecture
Go deeper into the stack.
Explore the complementary capabilities that can turn an individual technology into a complete workflow.
Your next move
Bring us the
real problem.
A workflow that takes too long. A system that cannot connect. An idea that needs a technical path. Start there, and we can define what to investigate, build and measure.
- A spatial, hands-busy or hazardous task
- 3D models or asset data to anchor
- Users and sites available for testing