Begin: Can start immediately

First Supervisor: Prof. Dr. Heisenberg

Second Supervisor: Natasha Randall MSc.

Level: Master's thesis

Problem:

Deep learning models can be black boxes, so explainable AI (XAI) techniques are important to help users understand why models make predictions, and to improve their interpretability. Shapley values [1] are a powerful method to compute feature importances, explaining why a particular model prediction was made.

In the examples below, from the SHAP (SHapley Additive exPlanations) implementation [2], the Shapley values reveal that the model's prediction of the price of a house was influenced most by the median income and its location. Shapley values can also be used for image data, to explain how different pixels or superpixels have contributed to the classification of an image, such as how each MNIST digit is classified as a number.

However, there is very little previous research on applying Shapley values to explain segmentation models. [3] used a SHAP inspired method to explain the segmentation of superpixels in terms of surrounding superpixels, and [4] applied Shapely values to highlight the contributions of different features for image classification, but there is still a big research gap at the intersection of generating Shapely-based feature importances for segmentation models.

Objective:

The objective of this thesis is to develop a methodology to apply Shapley values to image segmentation models to compute feature importances. This is extremely relevant to the segmentation of remote sensing imagery (such as segmenting flood waters from satellite images) where often many data features are input into the models (such as optical satellite imagery bands, radar satellite imagery, topographical elevation data, etc), and it is important to know which features were most relevant to the model.

Tasks:

• Develop a new algorithm that uses Shapley values to compute feature importances for segmentation models.
• Determine the best approach to generate the most useful explanations of the behaviour of deep learning models.
  • Combining feature importance and superpixels? Calculating importance per class? How best to calculate both global and local explanations?
• Evaluate the effectiveness of the developed method on real deep learning models.
  • Currently available are a flood prediction model and dataset, and a coffee/forest segmentation model and dataset.

If you are interested, please get in contact via email.