About
Traditional 2D imaging limits neurosurgeons’ ability to fully visualize and comprehend complex 3D structures. This challenge not only increases the risk of complications but also prolongs preoperative planning. !, addresses these issues by overlaying detailed 3D anatomical models directly onto the physical workspace in real time.
Challenge
Our team joined the project to turn the existing basic prototype into a functional high-performance product, addressing the following challenges:
ApplicationPerformance
The development of an AR application to seamlessly process high-load 3D models and DICOM files.
Connectivity andRendering
Implement a local WebRTC-based client-server rendering solution for stable offline processing.
Synchronization and Compatibility
Ensure that headset position, hand gestures, and event tracking are synchronized across multiple hardware platforms.
Optical TrackingReliability
Maintain continuous optical tool tracking for precise spatial synchronization during complex surgical movements.
Solution
To overcome the challenges identified and deliver a robust solution, it was required to develop a series of innovative approaches that go beyond standard practices. Each solution is carefully designed to address specific pain points while ensuring the system remains adaptable and future-proof, laying a strong foundation for continued innovation and reliability.
Optical Tool Tracking Integration
We leveraged QR code recognition and synchronization mechanisms to have surgical instruments aligned with spatial coordinates of Mixed Reality or Spatial Computing glasses and thus enhance overall operational accuracy.
Real-Time AR 3D Model Alignment
AR 3D model alignment functionality with 3D QR code markers was incorporated to accurately position virtual models in real-world environments and ensure continuous tracking.
Local Client-Server Rendering with WebRTC
To eliminate internet connection dependency, all application functionalities were transitioned to a desktop-based solution. The implementation of a WebRTC-based rendering engine allowed us to stream pixels directly to the Mixed Reality or Spatial Computing glasses over a local network.
Cross-Platform Hardware Compatibility
We achieved broader compatibility and future adaptability by decoupling hardware-specific functionalities through the transmission of positional and event data to a PC.
Technologies
Team and duration
10 experts
4 months
Qualium Systems is a reliable partner in building our Immersive Surgical Platform, both in Augmented Reality and in product development. They show flexibility and expertise in building MedTech product which is always challenging in terms of clinical application, safety and regulation. We were always on the same page in terms of communication and delivery. Thanks to them our startup can move faster and focus on what matters most.
