Ensemble of Deep 2D and 3D Fully Convolutional Neural Network for Brain Tumor Segmentation
- 0 Collaborators
Automated brain tumor segmentation using an ensemble of deep convolutional neural networks ...learn more
Project status: Published/In Market
Overview / Usage
We utilize an ensemble of the fully convolutional neural networks (CNN) for segmentation of gliomas and its constituents from multimodal Magnetic Resonance Images (MRI). The ensemble comprises of 3 networks, two 3-D and one 2-D network. Of the 3 networks, 2 of them (one 2-D & one 3-D) utilize dense connectivity patterns while the other 3-D network makes use of the residual connection. Additionally, a 2-D fully convolutional semantic segmentation network was trained to distinguish between air, brain, and lesion in the slice and thereby localize the lesion the volume. Lesion localized by the above network was multiplied with the segmentation mask generated by the ensemble so as to reduce the false positives. On the BraTS validation data (n=66), the scheme utilized in this manuscript achieved a whole tumor, tumor core and active tumor dice of 0.89 0.76, 0.76 respectively, while on the BraTS test data (n = 191), our scheme achieved the whole tumor, tumor core and active tumor dice of 0.83 0.72, 0.69 respectively.
Methodology / Approach
An ensemble of the fully convolutional neural network was utilized to segment gliomas and its constituents from multi-modal MR volume. The ensemble comprises of 3 networks ( two 3-D networks and one 2-D network). Two networks ( a 3-D and a 2-D network) utilizes dense connectivity patterns while the other 3-D network comprises of residual connection. The networks with dense connectivity pattern were semantic segmentation networks and predict the class associated with all pixels or voxels that form the input to the network. The network with residual connectivity pattern was composed of inception modules so as to learn multi-resolution features. This multi-resolution network unlike the other networks in the ensemble classifies only a subset of voxels.
A 2-D fully convolutional semantic segmentation (Air-Brain-Lesion Network) was trained to delineate air, brain, and lesion from the axial slice of the MR volumes and thereby localize the lesion in the volume. The predictions generated by the ensemble were smoothened by using Conditional random fields. The smoothened prediction and the output generated by the Air-Brain-Lesion network were used in tandem to reduce the false positives in the prediction. The false positives in the predictions were further reduced by incorporating a class-wise 3-D connected component analysis in the pipeline.
Technologies Used
Deep learning, Computer Vision, Pytorch, matplotlib, simpleITK, numpy, sklearn, and pandas