arXiv Analytics

Sign in

arXiv:2102.13645 [eess.IV]AbstractReferencesReviewsResources

Convolution-Free Medical Image Segmentation using Transformers

Davood Karimi, Serge Vasylechko, Ali Gholipour

Published 2021-02-26Version 1

Like other applications in computer vision, medical image segmentation has been most successfully addressed using deep learning models that rely on the convolution operation as their main building block. Convolutions enjoy important properties such as sparse interactions, weight sharing, and translation equivariance. These properties give convolutional neural networks (CNNs) a strong and useful inductive bias for vision tasks. In this work we show that a different method, based entirely on self-attention between neighboring image patches and without any convolution operations, can achieve competitive or better results. Given a 3D image block, our network divides it into $n^3$ 3D patches, where $n=3 \text{ or } 5$ and computes a 1D embedding for each patch. The network predicts the segmentation map for the center patch of the block based on the self-attention between these patch embeddings. We show that the proposed model can achieve segmentation accuracies that are better than the state of the art CNNs on three datasets. We also propose methods for pre-training this model on large corpora of unlabeled images. Our experiments show that with pre-training the advantage of our proposed network over CNNs can be significant when labeled training data is small.

Related articles: Most relevant | Search more
arXiv:2004.10792 [eess.IV] (Published 2020-04-22)
Automatic Polyp Segmentation Using Convolutional Neural Networks
arXiv:2006.13212 [eess.IV] (Published 2020-06-23)
Automated Detection of COVID-19 from CT Scans Using Convolutional Neural Networks
arXiv:1904.04671 [eess.IV] (Published 2019-04-07)
Surface Defect Classification in Real-Time Using Convolutional Neural Networks