Virtual Reality, Telepresence, and Perception Engineering
In this research, we view virtual reality (VR) as a problem of perception
engineering, which requires the design, development, and delivery
of a perceptual illusion through artificial stimulation of the human
senses. Each human sense is capable of such illusions; in the case of
vision, we qare familiar with many optical illusions. Because VR
directly impacts the human body and even disrupts its ordinary
functions, it is crucial to understand human physiology, neuroscience,
and perception and how they respond to VR technology. We strive to
determine perception-based and physiology-based criteria that capture
important qualities such as task effectiveness, comfort, and
presence. These criteria are used to guide the engineering
specifications for VR systems. Typical challenges are viewpoint
movement, display artifacts, rendering methods, interaction
mechanisms, distributed computation, and limitations of wireless
communication.
Perception engineering involves the forward engineering of VR systems
with the tight integration of low-level human considerations, which
are essentially obtained via reverse engineering (we did not engineer
ourselves). Based on current academic fields, this requires a highly
interdisciplinary approach; however, one day perception engineers
might emerge, who are specifically trained in methodologies based on
the science of perception. This would follow a similar path as
existing engineering fields. Civil, mechanical, and electrical
engineering derive from physics. Chemical and biological engineering
derive from chemistry and biology, respectively. Likewise, perception
engineering should derive from perceptual psychology and related
fields of physiology, medicine, and neuroscience, while also building
upon existing engineering principles.
Some highlights are:
- We developed a general mathematical theory of what VR actually is, providing a potential foundation for the field that is analogous to Turing machines for computer science or xdot=f(x,u) for control theory (AR CRAS 2024). We define perception engineering (and VR) as one agent intentionally altering the perceptual experience of another, while maintaining plausibility and illusions. The theory applies both to biologicial organisms and fully engineered systems, such as robots.
- Using EEG methods, we obtained a deeper understanding of how VR sickness affects VR user performance (TVCG 2023).
- We introduced techniques to improve the experience of robotic tele-embodiment using VR (HRI 2022, IROS 2022, and ISMAR 2022).
- We established that people have an extreme misunderstanding of how physics behaves at smaller (or larger) scales (FVR 2021 and TVCG 2023).
- We wrote a Virtual Reality textbook, freely available online, published in 2023 by Cambridge University Press, and used in courses around the world.
- We developed the original VR head tracking method (ICRA 2014) and its software implementation for the Oculus Rift, where it was deployed until purchased by Facebook in 2014 (and possibly used thereafter).
Papers on Virtual Reality, Telepresence, and Perception Engineering
Adaptation to simulated hypergravity in a virtual reality throwing task. M. Pouke, E. Uotila, E. G. Center, K. G. Timperi, A. P. Chambers, T. Ojala, and S. M. LaValle. ACM Transactions on Applied Perception, 2024. [pdf].
From virtual reality to the emerging discipline of perception engineering. S. M. LaValle, E. G. Center, T. Ojala, M. Pouke, N. Prencipe, B. Sakcak, M. Soumalainen, K. G. Timperi, and V. Weinstein. Annual Reviews on Control, Robotics, and Autonomous Systems, 2024. [pdf].
Virtual reality sickness reduces attention during immersive experiences. K. J. Mimnaugh, E. G. Center, M. Suomalainen, I. Becerra, E. Lozano, R. Murrieta-Cid, T. Ojala, S. M. LaValle, and K. D. Federmeier. IEEE Transactions on Visualization and Computer Graphics, 29(11):4394-4404, 2023. [pdf].
Virtual Reality. S. M. LaValle. Cambridge University Press, 2023. Also available at http://lavalle.pl/vr/, [pdf].
Leaning-based control of an immersive-telepresence robot. J. Halkola, M. Suomalainen, B. Sakcak, K. J. Mimnaugh, J. Kalliokoski, A. P. Chambers, T. Ojala, and S. M. LaValle. In IEEE International Symposium on Mixed and Augmented Reality, 2022. [pdf].
HI-DWA: human-influenced dynamic window approach for shared control of a telepresence robot. J. Kalliokoski, M. Suomalainen, B. Sakcak, A. P. Chambers, T. Ojala, and S. M. LaValle. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2022. [pdf].
An enactivist-inspired mathematical model of cognition. V. Weinstein, B. Sakcak, and S. M. LaValle. Frontiers in Neurorobotics, September 2022. [pdf].
The body scaling effect and its impact on physics plausibility. M. Pouke, E. G. Center, A. P. Chambers, S. Pouke, T. Ojala, and S. M. LaValle. Frontiers in Virtual Reality, May 2022. [pdf].
Augmenting immersive telepresence experience with a virtual body. N. Arora, M. Suomalainen, M. Pouke, E. G. Center, K. J. Mimnaugh, A. P. Chambers, S. Pouke, and S. M. LaValle. IEEE Transactions on Visualization and Computer Graphics, 28(5):2135-2145, 2022. [pdf].
Human perception engineering. E. G. Center, K. Mimnaugh, J. Hakkinen, and S. M. LaValle. In M. Alcaniz, M. Sacco, and J. G. Tromp, editors, Roadmapping Extended Reality: Fundamentals and Applications. John Wiley \& Sons, 2022. [pdf].
Unwinding rotations improves user comfort with immersive telepresence robots. M. Suomalainen, B. Sakcak, A. Widagdo, J. Kalliokoski, A. P. Chambers, T. Ojala, and S. M. LaValle. In ACM/IEEE International Conference on Human-Robot Interaction, pages 411-520, 2022. [pdf].
The plausibility paradox for resized users in virtual environments. M. Pouke, K. J. Mimnaugh, A. P. Chambers, T. Ojala, and S. M. LaValle. Frontiers in Virtual Reality, April 2021. [pdf].
Analysis of user preferences for robot motions in immersive telepresence. K. J. Mimnaugh, M. Suomalainen, I. Becerra, E. Lozano, R. Murrieta-Cid, and S. M. LaValle. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2021. [pdf].
Assessing postural instability and cybersickness through linear and angular displacement. C. J. Widdowson, I. Becerra-Duran, C. Merrill, J. Wang, and S. M. LaValle. The Journal of the Human Factors and Ergonomics Society, 63(2):296-311, 2021. [pdf].
Comfort and sickness while virtually aboard an autonomous telepresence robot. M. Suomalainen, K. J. Mimnaugh, I. Becerra, E. Lozano, R. Murrieta-Cid, and S. M. LaValle. In P. Bourdot, M. Alcaniz Raya, P. Figueroa, V. Interrante, T. W. Kuhlen, and D. Reiners, editors, EuroXR 2021: Virtual Reality and Mixed Reality (Lecture Notes in Computer Science, 13105), pages 3-24. Springer-Verlag, Berlin, 2021. [pdf].
The plausibility paradox for scaled-down users in virtual environments. M. Pouke, K. J. Mimnaugh, T. Ojala, and S. M. LaValle. In 27th IEEE Conference on Virtual Reality and 3D User Interfaces, pages 913-921, 2020. [pdf].
Human perception-optimized planning for comfortable VR-based telepresence. I. Becerra, M. Suomalainen, E. Lozano, K. J. Mimnaugh, R. Murrieta-Cid, and S. M. LaValle. IEEE Robotics and Automation Letters, 5(4):6489-6496, 2020. [pdf].
Virtual reality for robots. M. Suomalainen, A. Q. Nilles, and S. M. LaValle. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 11458-11465, 2020. [pdf].
Can simulated nature support health? Comparing short, single-doses of 360-degree nature videos in virtual reality with the outdoors. M. H. E. M. Browning, K. J. Mimnaugh, C. J. van Riper, H. K. Laurent, and S. M. LaValle. Frontiers in Psychology, 2020. [pdf].
Efficacy study on interactive mixed reality (IMR) software with sepsis prevention medical education. N. K. Sankaran, H. J Nisar, J. Zhang, K. Formella, J. Amos, L. T. Barker, J. Vozenilek, S. M. LaValle, and T. Kesavadas. In IEEE Conference on Virtual Reality and 3D User Interfaces, 2019. [pdf].
Effects of visual realism and moving detail on cybersickness. M. Pouke, A. Tiiro, S. M. LaValle, and T. Ojala. In IEEE Conference on Virtual Reality and 3D User Interfaces, 2018. [pdf].
Virtual reality visualization of patient specific heart model. M. Bramlet, K. Wang, A. Clemons, N C. Speidel, S. M. LaValle, and T. Kesavadas. Journal of Cardiovascular Magnetic Resonance, 18(Suppl 1):T13, 2016. [pdf].
Head tracking for the Oculus Rift. S. M. LaValle, A. Yershova, M. Katsev, and M. Antonov. In IEEE International Conference on Robotics and Automation, 2014. [pdf].