AKTIS is a room-scale, two-player colocated MR experience on Meta Quest. In a shared physical room, players hunt for everyday objects and manipulate real props to solve networked narrative puzzles — with magical moments when virtual content responds to what happens in the real world. Passthrough vision tracks handheld markers and Photon Fusion keeps both players in sync in one colocated session.
Instead of traditional controllers, AKTIS explores the idea that everything in the real world has the potential to become an interface. Two players share one physical room — moving, collaborating, and experimenting together while handheld tangible props tracked through passthrough ArUco vision drive networked narrative puzzles.
When virtual events respond back to physical space, reality and virtuality feel deeply intertwined — creating surprising, memorable magical moments.
Play with sound — includes live playtester reactions.
AKTIS connects physical props and room geometry to a shared colocated MR session on Meta Quest. Handheld objects with ArUco markers feed the passthrough API into per-headset, multi-threaded marker detection (OpenCV, pose filtering); the scanned room flows through Meta MRUK into spatial-anchored environment props. Both paths converge at a host-authoritative network sync layer (Photon Fusion) that replicates shared objects so two players manipulate the same tangible interfaces in one physical room.
Under the hood, ScriptableObject event channels decouple CV, spawn, quest, and puzzle code — click Event Channels in Technical Pillars below for how local detections become host-owned networked state without cross-references.
Built on Unity 6, Meta Quest, Photon Fusion 2, and OpenCV ArUco.
Gameplay systems communicate through ScriptableObject event channels rather than direct references — DetectedMarkerEventChannelSO for local CV, ARObjectInfoEventChannelSO for spawn requests, AnchorObjectEventChannelSO for environment props, and IntEventChannelSO / VoidEventChannelSO for quest and puzzle completion.
Publishers call .Invoke(payload); subscribers wire onEvent in the inspector or Spawned(). With 10+ systems across CV, networking, quests, NPCs, and puzzles, channels let both engineers iterate in parallel without circular scene references or cross-subsystem coupling.
Built on Yoshinaga’s Quest passthrough + ArUco OpenCV baseline, AKTIS replaces the single-threaded update loop with an async pipeline (AsyncGPUReadback + background OpenCV) tuned for two-player Fusion VR frame times.
Quest passthrough frames enter ArucoLocalManager, which drives CustomArucoDetector for marker detection and pose filtering. Detections publish via DetectedMarkerEventChannelSO, keeping the CV layer independent from networking — see Host-authoritative Fusion below for how local observations become host-owned spawn state.
I scoped colocation as a separate workstream from marker tracking; teammate Chih-I Chou researched and implemented shared physical-space alignment via OVRColocationSession and MRUK.ShareRoomsAsync. The host advertises a colocation session, shares room UUIDs, and syncs remote floor pose over Fusion [Networked] properties before any shared content appears.
Once both headsets share the host’s MRUK-aligned space, fixed environmental content spawns from Meta Quest Environment Setup anchor transforms — separate from handheld marker-tracked props. EnvironmentSpawnNetworkObjectRequest fires per anchor when Fusion is ready; the host spawns NetworkObject instances at each pose via AnchorObjectEventChannelSO, preserving ArUco tracking budget for movable tangible interfaces.
Clients report local marker observations through NetworkARObjectReporter, which emits ARObjectInfo on ARObjectInfoEventChannelSO. ArUcoNetworkManager on the Fusion host listens for those requests, owns the spawn registry, and applies updates via [Rpc(RpcSources.All, RpcTargets.StateAuthority)] — centralizing spawn decisions so both players see the same tangible props.
Marker-gated visuals are opt-in, not the default for every spawned prop. Drawer surprises use NetworkObjectHideVisual with _disappearIfNotSeen on NetworkARObjectReporter: a networked Visable flag drives showEvent / hideEvent on each client, so virtual effects stay dormant until passthrough actually sees the marker. Open the drawer, spot the marker on ordinary office paper, and the enchantment triggers — sheets fly out one after another. Both players share the same reveal because the Fusion host syncs visibility.
Click to learn more…
Click to learn more…
Click to learn more…
Click to learn more…
Click to learn more…
Click to learn more…