In The End

I started developing this prototype during the summer, implementing the elastic band stretching effect through code. Since the haptic gloves hadn’t arrived yet, I initially prototyped using the built-in haptics of Meta Quest controllers. Once the gloves arrived, thanks to the modular Haptic System I had established, I only needed to swap out the hardware interface layer—all the upper-level software implementation remained unchanged, allowing seamless transition to glove-based haptic playback.

User feedback on this prototype was positive—players enjoyed the tension effect when stretching the elastic band. I designed the haptic feedback in three distinct phases: grabbing the band, stretching it, and the sharp ‘snap’ impact upon release. Testing confirmed that adding haptics increased user immersion, though the emotional impact wasn’t particularly pronounced.

In this interaction, I explored multiple dimensions:

1. Ray-cast detection: Using ray-cast to detect contact points, so that individual fingertips trigger vibrations based on whether they are touching an object.
2. Dynamic intensity: Weakening or strengthening haptic feedback based on finger distance to simulate light vs. firm strokes.
3. Indirect control: Experimenting with how haptic feedback can serve as negative feedback.

One of the core design challenges in VR is that virtual objects have no physical presence…

No Way to Stop the Hand

You cannot physically prevent a user’s hand from “passing through” an object. How to communicate to users what the “correct” action is, especially for something as nuanced as stroking, is a genuinely interesting topic.

Can Haptics Say “Wrong”?

From the early exploration phase, we found that shock-like haptic sensations caused users to recoil and feel discomfort — though at that point, we had not yet investigated whether this discomfort would actually lead users to avoid such contact. This gave rise to an initial hypothesis: that an extremely intense haptic feedback could be used to convey a signal of “wrong” or erroneous behavior.

More Rock, Less Talk

Based on this hypothesis, I designed an interaction in which, when a user touches the “interior” of an object (i.e., passes beyond its surface), the haptic feedback intensity increases sharply, prompting users to notice the discomfort and withdraw their hand.

What the Playtesters Taught Me

After having playtesters try it, the results were not significant. Users did not interpret intense haptic feedback as an error signal, and the perceived intensity of the haptics varied considerably across age groups — for example, Playtester A found the vibration very strong, while Playtester B found it barely noticeable. This led me to understand that normalizing haptic feedback is inherently difficult, and that people’s interpretation of haptics requires further research before it can reliably serve as a feedback guide.

This experiment explored impact-based haptics through balloon slapping. I implemented floaty balloon physics that reacted naturally to hits while maintaining upward drift. Users found the ‘pop’ satisfying and instinctively continued interacting.

Our Sound Designer, Michaecle, added switchable audio—realistic vs. cartoon. The emotional shift was dramatic: cartoon sounds transformed the experience into something playful and laughter-inducing.

While engaging, the haptic impact on emotion still needed strengthening.

In this prototype, I primarily tested whether the sensitivity of collision detection and haptic triggering was sufficient. The results showed that single-point collision is viable; however, the limitation still lies in Unity’s physics engine detection quality — if the raindrop moves too fast, it cannot be detected. On the visual side, I also found an interesting insight about general perception: most people associate raindrops with a nearly linear shape, which was a notable finding about common mental models.

This prototype explores how individuals conceptualize abstract phenomena. Our findings reveal that mental models vary significantly: some users envision subtle vibrations, while others imagine energy flowing from their fingertips to their wrists. This diversity highlights a key challenge in hand gesture prediction—since there is no universal ‘standard’ for telekinesis, users naturally employ a wide range of intuitive gestures. This experience underscored that for abstract, non-standardized mechanics (unlike the well-established ‘magic wand’ trope), continuous playtesting and behavioral analysis are indispensable.

How would you pinch your father?’ This prototype investigates how social context and empathy reshape physical interactions with virtual entities. Beyond testing haptic feedback for the ‘pinch’ gesture, we examined how shifting the identity of a virtual character—assigned via dynamic text labels—altered user behavior. Our study revealed that users were reluctant to use a pinch gesture on figures of authority, such as a ‘King,’ preferring instead to support them in their palms. Furthermore, many participants expressed moral hesitation to pinch virtual humans altogether. These findings underscore that empathy and social hierarchy significantly dictate interaction paradigms, suggesting that context is essential for creating truly intuitive and culturally resonant XR experiences.

For this prototype, we initially used the Inspector to adjust the heartbeat frequency, then asked players how they would interpret each variation and how it made them feel. This testing process revealed that heartbeat carries a broadly shared meaning across users, and because it is so closely tied to life itself, it tends to evoke emotion more readily. As a result, this interaction became the centerpiece of our experience.

This prototype required the use of the Interactor module from the Meta SDK. Since the module didn’t offer fine-grained control over finger curl degree for gesture detection, we went deeper into the Meta SDK and modified the relevant source code directly to make the interaction possible. The biggest challenge, however, was optimizing for the wide variation in how people crush objects — both in terms of speed and motion range. One key issue we encountered was that users with prior VR experience would instinctively “grab” or “pinch” objects, rather than naturally conforming their hand shape to the object as they would in real life. To handle this, we implemented several nuanced conditions: for instance, if a user’s fingers were already beyond the curl threshold upon first contact with the object, the Crush interaction would not trigger immediately — instead, it would wait until the user opened their hand again before resuming detection. These refinements ensured that users could fully experience the haptic feedback throughout the crush motion, and this logic was carried forward into our final Crush Heart interaction.