| Creators: |
Hieber, Daniel and Karthan, Maximilian and Holl, Felix and Prokop, Georg and Märkl, Bruno and Pryss, Rüdiger and Liesche-Starnecker, Friederike and Schobel, Johannes |
| Title: |
Evaluating the Segment Anything Model for Histopathological Tissue Segmentation |
| Item Type: |
Conference or Workshop Item |
| Event Title: |
68. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS) |
| Event Location: |
Heilbronn, Germany |
| Event Dates: |
17.-21. September 2023 |
| Projects: |
DigiHealth, NAP |
| Page Range: |
DocAbstr. 159 |
| Date: |
2023 |
| Divisions: |
Gesundheitsmanagement |
| Abstract (ENG): |
Introduction: Meta introduced the Segment Anything Model (SAM), a new Machine Learning model for segmentation with zero-shot generalization and no need for additional training data, in April 2023 [1]. Since then, the model has already been applied to medical imaging, such as skin cancer segmentation [2] and brain extraction from MRI images [3]. However, no work addresses the tile-wise segmentation approach, which is the current state of the art for histopathological segmentation.
Methods: A histopathological dataset of 103 labeled Whole-Slide-Images (WSIs) of Hematoxylin and Eosin (HE) stained Glioblastoma (GBM) slices from the Technical University of Munich was selected for the segmentation task. The labeling was conducted by domain experts (i.e., neuropathologists). GBM prove an especially hard segmentation task, due to their high heterogeneity and fluid borders. The WSIs were sliced into tiles with a resolution of 1024x1024 pixels, which were further rescaled to 256x256 pixels to match the input of the reference model used for comparison.
The selected tiles were then annotated using SAM’s point labeling before training. Three different methods were evaluated for the annotation process. 1) manual annotation using 20 prompts and automatic annotation based on the original label of the dataset using a 2) grid-based strategy (prompts equally distributed over the image in a grid) and 3) border-based strategy (labeling the border between tumor and tissue). The results were then compared with a U-Net specifically trained on the given dataset [4].
Results: The automatic labeling approaches, and a bounding box-based approach, did not provide any usable segmentation output. The manual labeling resulted in an Intersection-over-Union (IoU) score close to ~40% compared to an IoU of 84.5% using the U-Net approach.
Using other resolutions for the input (e.g., 1024x1024 pixels without rescaling) did not provide any benefit and resulted in IoU scores of ~30%.
Discussion: The results indicate that SAM is currently incapable of segmenting histopathological images out of the box. This finding is in accordance with the work of Deng et al., who tested SAMs histopathological segmentation capabilities on heavily rescaled, non-tiled WSIs of skin cancer, which is a relatively easy tumor to segment [2]. As with any other model, fine-tuning and additional training are required to make SAM suitable for histopathological image segmentation. This removes SAM’s most significant advantage, zero-shot segmentation. Results for other staining methods (e.g., immunohistochemical staining) could provide different results, although this is unlikely, as indicated by [2].
Conclusion: While SAM shows competitive results for other medical segmentation tasks (i.e., nuclei segmentation [2] or MRI segmentation [3]), its histopathological tissue segmentation capabilities using a zero-shot approach are not comparable to an established and well-trained state-of-the-art model. While SAM’s point annotations could assist domain experts in the initial dataset labeling, it is not suitable in its current form for the final segmentation task. In first tests, it was able to detect half the tumor with three annotations, already dramatically reducing the time needed for labeling.
Future work has to evaluate SAM’s trained segmentation capabilities compared to current state-of-the-art models. |
| Forthcoming: |
No |
| Language: |
English |
| Link eMedia: |
Download |
| Citation: |
Hieber, Daniel and Karthan, Maximilian and Holl, Felix and Prokop, Georg and Märkl, Bruno and Pryss, Rüdiger and Liesche-Starnecker, Friederike and Schobel, Johannes
(2023)
Evaluating the Segment Anything Model for Histopathological Tissue Segmentation.
In: 68. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS), 17.-21. September 2023, Heilbronn, Germany, DocAbstr. 159.
|
