This research, supported by NCC Montenegro and NCC Bosnia and Herzegovina team members through EuroCC2 and EuroCC4SEE, investigates how Explainable Artificial Intelligence (XAI) can strengthen trust and transparency in medical AI systems. By applying XAI to predictive models for respirator maintenance, the work demonstrates how high-performance computing (HPC) enables more reliable, interpretable, and clinically useful AI solutions.
Challenge / Problem
Machine learning models used in medicine often function as “black boxes,” offering accurate predictions but without clear insight into how decisions are made. In safety-critical areas—such as predicting failures in medical respirators—this lack of transparency limits acceptance, slows adoption, and poses risks to clinical safety and regulatory compliance. Interpreting these models requires computationally intensive methods that exceed standard workstation capabilities.
Solution
The study applied SHAP and LIME to interpret machine learning models trained on a large dataset of 75 medical respirators and 1,350 performance records. Using HPC infrastructure provided by NCC Montenegro, the student was able to efficiently train models, compute global and local explanations, compare interpretability methods, and validate the factors most responsible for predicting device malfunction. The HPC-enabled workflow ensured faster experimentation, reproducibility, and scalable analysis suitable for medical applications. This work resulted in Master thesis defence by Ms Ivana Lalatovic at University of Donja Gorica and was additionally supported by Verlab Institute that provided a real-life industry dataset.
Benefits
- Increased transparency of AI predictions through detailed SHAP and LIME explanations
- Faster model training and interpretation made possible by HPC resources
- Higher trust and reliability in AI-assisted medical decision systems
- A reusable XAI methodology applicable to other healthcare datasets and diagnostic tasks