| Augmented reality (AR) technologies enable users to interact with virtual content in fundamentally new ways. AR applications capture input from a user’s surroundings, such as video, depth sensor data, or audio, and they overlay output (for instance, visual, audio, or haptic feedback) directly on the user’s perception of the real world, through devices like smartphones, head-mounted displays (HMDs), or automotive windshields. While the vision of AR is decades old,1 AR technologies are only now on the cusp of commercial viability and beginning to capture the attention of users worldwide. From the wildly popular mobile AR app Pokémon Go to powerful HMDs like Microsoft’s HoloLens and Meta’s Meta2, as well as AR-enabled car windshields2 and military applications,3 interest in AR technologies across diverse industry sectors is increasing. Figure 1 shows two examples. Although AR technologies promise great potential benefits, they also raise new and serious computer security and privacy risks. For example, AR applications’ need for rich, continuous sensor data (such as video and audio feeds) raises privacy concerns for both users and bystanders. The ability for AR applications to generate virtual (visual, audio, or haptic) content that modifies a user’s perception of the physical world also raises new security and safety risks. For example, consider a buggy or malicious AR windshield application that obscures real-world pedestrians, overlays misleading information on real-world road signs, startles the user while driving, or strategically obstructs virtual content from another, simultaneously running AR application. Addressing AR output risks is particularly critical for fully immersive AR systems, such as HMDs and car windshields, where users cannot easily disengage from their devices if output security issues arise. Figure 2 shows an abstract architecture of an AR platform, with sensor input coming in and virtual content produced as output. The academic computer security community has begun turning its attention to the potential input and output risks with AR4—focusing primarily on risks from buggy or malicious applications rather than the AR platform itself—and exploring potential solutions to mitigate these risks. On the input side, prior efforts have studied user perception of AR privacy risks5 and worked to mitigate these risks, for instance, by limiting the amount of sensor data made available to AR applications6–۸ or by enforcing context-based policies on sensor data collection.9,10 (This and additional related work is discussed further in the conference version of this article.11) However, little work has considered risks or mitigations on the output side. |
