Real-time Kidney Ultrasound with AR and Deep Learning

Ultrasound is fast, safe, and affordable. Doctors often scan kidneys to track disease, and kidney size is a key marker. Measuring it during a live scan takes time, breaks focus, demands expertise, and results can vary from one person to another. Ultrasound images also change from moment to moment, so any helpful AI needs to work in real time. On top of that, clinicians keep looking back and forth between the patient and a monitor, which hurts ergonomics and rapport.

We combine real-time AI that automatically identifies the kidney and measures width, length, and volume. The application works on an augmented reality headset; thus, the sonographer sees the live ultrasound and measurements directly in their view. This reduces head turns, accelerates exams, and ensures more consistent measurements. Our setup is compatible with most ultrasound machines via two connection options, and we evaluated its accuracy and functionality using both public datasets and prospectively acquired data from healthy subjects.

Publication

Luijten, G., Scardigno, R. M., Faray de Paiva, L., Hoyer, P., Kleesiek, J., Buongiorno, D., Bevilacqua, V., Egger, J. (2025). Deep Learning-Based Semantic Segmentation for Real-Time Kidney Imaging and Measurements with Augmented Reality-Assisted Ultrasound. arXiv:2506.23721.

Partners

• Institute for Artificial Intelligence in Medicine (IKIM)
  • Gijs Luijten
  • Lisle Faray de Paiva
  • Prof. Jens Kleesiek
  • Prof. Jan Egger
• Department of Electrical and Information Engineering (DEI), Polytechnic University of Bari
  • Roberto Maria ScardignoProf. Domenico Buongiorno
  • Prof. Vitoantonio Bevilacqua
• Pediatric Clinic II
  • Prof. Peter Hoyer

Contact

Gijs Luijten (Gijs.Luijten@uk-essen.de)