Mind the GAP: Geometry Aware Passthrough Mitigates Cybersickness

Abstract

Virtual Reality headsets isolate users from the real-world by restricting their perception to the virtual-world. Video See-Through (VST) headsets address this by utilizing world-facing cameras to create Augmented Reality experiences. However, directly displaying camera feeds can cause visual discomfort and cybersickness due to the inaccurate perception of scale and exaggerated motion parallax. This paper presents initial findings on the potential of geometry aware passthrough (GAP) systems to mitigate cybersickness through enhanced depth perception. We introduce a promising protocol for quantitatively measuring cybersickness experienced by users in VST headsets. Using this protocol, we conduct a user study to compare direct passthrough (DP) and geometry aware passthrough systems. To the best of our knowledge, our study is the first one to reveal reduced nausea, disorientation, and total scores of cybersickness with geometry aware passthrough. It also uncovers several potential avenues to further mitigate visually-induced discomfort.

Passthrough Modes

2D illustration comparing DP and GAP reprojection

2D illustration comparing reprojection between DP and GAP: The foreground and background are reprojected using different geometries. In DP, each 3D point is assumed to belong to a single plane situated at a certain distance. This results in reprojecting objects at incorrect pixel locations and scale (left). In comparison, GAP uses a geometry estimate which improves the perceived location and scale (right).

Comparison of DP and GAP outputs showing distortions and scaling

DP versus GAP output images: Two images taken from the HMD at the same pose. DP enlarges objects, making the scene look closer to the user. The scale difference can be noticed when comparing the text on the bottom right of both images. While GAP improves the scale, it results in warping artifacts which can be observed on the edge of the door behind the hand.

User Study

User Study Setup: Pictures of the lab setup for the tasks completed by participants while wearing the VST HMD.

SSQ: Box plots of Nausea (N), Oculomotor (O), Disorientation (D), and Total subscores of simulator sickness comparing all conditions. Natural Vision (NV) led to significantly lower N, O, D, and total sickness scores than both DP and GAP (p < 0.05). GAP led to significantly lower N (p = 0.016), D (p = 0.029), and total (p = 0.011) sickness scores than DP.

Subjective Discomfort: Box plots of discomfort scores for all conditions and preference for passthrough conditions across the typing, navigation, and interaction tasks. NV led to significantly lower discomfort scores than DP and GAP (p < 0.05) across all tasks. Compared to DP, GAP led to significantly lower discomfort scores across all tasks, typing (p = 0.046), navigation(p = 0.041), interaction (p = 0.022).

BibTeX

@inproceedings{10.1145/3706599.3720163, author = {El Chemaly, Trishia and Goyal, Mohit and Duan, Tinglin and Phadnis, Vrushank and Khattar, Sakar and Vlaskamp, Bjorn and Kulshrestha, Achin and Turner, Eric Lee and Purohit, Aveek and Neiswander, Gregory and Tsotsos, Konstantine}, title = {Mind the GAP: Geometry Aware Passthrough Mitigates Cybersickness}, year = {2025}, booktitle = {Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems} }

Mind the GAP: Geometry Aware Passthrough Mitigates Cybersickness

Abstract

Passthrough Modes

User Study

User Study Setup: Pictures of the lab setup for the tasks completed by participants while wearing the VST HMD.

Video Presentation

CHI 2025 Poster

BibTeX

Acknowledgements