Leveraging Natural Human Behavior for Efficient Embodied Intelligence in Wearable Computing Systems

Abstract: Wearable computing systems, including augmented and virtual reality (AR/VR) devices, are emerging as a new generation of platforms that tightly integrate digital intelligence with the physical world. Looking forward, these devices are expected to evolve beyond passive displays into embodied intelligent agents that continuously perceive the environment, understand user intent and provide timely assistance across applications such as autonomous scientific research, professional training, healthcare and everyday productivity.

However, enabling embodied intelligence on wearable devices is challenging due to strict constraints on latency, energy, memory and computation. A key opportunity is that wearable systems maintain a direct and continuous interface with the human user. Natural behaviors such as gaze, head motion, hand interaction and attention reveal what information is most relevant and can be used to guide system-level optimization.

In this talk, I will present recent progress from my group on efficient wearable computing for embodied artificial intelligence. The talk will cover systems and hardware techniques that leverage human behavioral signals to support efficient perception, tracking, graphics and embodied intelligence under tight device constraints. Together, these efforts illustrate how human-centered system design can enable wearable platforms to continuously sense, reason and respond to the user and physical world in an efficient, intelligent and responsive manner.

Bio: Sai Qian Zhang is an assistant professor of electrical engineering and computer science at New York University (NYU). Prior to joining NYU, he spent two years at Reality Labs at Meta, working on smart sensor design for next-generation AR/VR devices. Sai earned his Ph.D. in computer science from Harvard University in 2021 and holds M.Sc. and B.Sc. degrees in electrical engineering from the University of Toronto.