Multimodal markers for technology-independent integration of augmented reality devices and surgical navigation systems

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

Augmented reality (AR) permits the visualization of pre-operative data in the surgical field of view of the surgeon. This requires the alignment of the AR device’s coordinate system with the used navigation/tracking system. We propose a multimodal marker approach to align an AR device with a tracking system: in our implementation, an electromagnetic tracking system (EMTS). The solution makes use of a calibration method which determines the relationship between a 2D pattern detected by an RGB camera and an electromagnetic sensor of the EMTS. This allowed the projection of a 3D skull model on its physical counterpart. This projection was evaluated using a monocular camera and an optical see-through device (HoloLens 2) (https://www.microsoft.com/en-us/hololens/) achieving an accuracy of less than 2.5 mm in the image plane of the HoloLens 2 (HL2). Additionally, 10 volunteers participated in a user study consisting of an alignment task of a pointer with 25 projections viewed through the HL2. The participants achieved a mean error of 2.7 1.3 mm and 2.9 2.9 in positional and orientation error. This study showcases the feasibility of the approach, provides an evaluation of the alignment, and finally, discusses its advantages and limitations.

Original languageEnglish
JournalVirtual Reality
DOIs
Publication statusAccepted/In press - 2022

Bibliographical note

Publisher Copyright:
© 2022, The Author(s).

Fingerprint

Dive into the research topics of 'Multimodal markers for technology-independent integration of augmented reality devices and surgical navigation systems'. Together they form a unique fingerprint.

Cite this