Spare Theory for the Detection of Brain Tumor using Multimodal Medical Image Fusion

S. L. Jany Shabu, C. Jayakumar, G. Arulselvi, D. Poornima, J. Refonaa, S. Dhamodaran


The fusion of multimodal images is a trending research area, especially in the field of medical image processing. The purpose of image fusion is to classify medical images efficiently. The objective of the research work is to do the fusion of multimodal medical images for doing medical image classification. In this research, a new algorithm is proposed for the detection of brain tumors based on three main steps namely, fusion, segmentation, and classification. A sparse theory-based vector selection (STVS) algorithm is proposed for image fusion. In this algorithm, the multimodal images are first converted into patches. These patches are further vectorized. The vectorized patches are employed in the creation of dictionaries. The generated dictionaries along with the vectorized patches are used for the creation of sparse matrices. From the sparse matrices, a selection vector is formed using which the fused image is generated. The segmentation of the fused image is done using Intuitionistic fuzzy set-based k-means (IFSKM) clustering and the Otsu thresholding technique. The clusters of the IFSKM are generated based on the Intuitionistic fuzzy set (IFS) scheme. Finally, classification is performed based on a DCNN architecture. The proposed system is validated using the brain images from the Harvard Medical School. Quantitative analysis reveals that the proposed scheme achieves the best performance in terms of fusion, segmentation, and classification. The proposed STVS scheme attained high values of entropy, standard deviation, PSNR in dB, mean square error (MSE), structural similarity index (SSIM), and homogeneity with the values of 7.33, 55.25, 42.85, 0.098, 64.31, and 53.52 respectively.


Keywords: multimodal, fusion, segmentation, intuitionistic fuzzy set, structural similarity index, classification.


Full Text:



JALALI V., & KAUR D. A study of classification and feature extraction techniques for brain tumor detection. International Journal of Multimedia Information Retrieval, 2020, 9(4): 271–290.

SABA T., SAMEH MOHAMED A., EL-AFFENDI M., AMIN J., and SHARIF M. Brain tumor detection using fusion of hand crafted and deep learning features. Cognitive. Systems Research, 2020, 59: 221–230,

SHARIF M., AMIN J., RAZA M., ANJUM M. A., AFZAL H., and SHAD S. A. Brain tumor detection based on extreme learning. Neural Computing and Applications, 2020, 32(20): 15975–15987.

BAI X., ZHOU F., and XUE B. Fusion of infrared and visual images through region extraction by using multi scale center-surround top-hat transform. Optics Express, 2011, 19(9): 8444.

AMIN J., SHARIF M., RAZA M., SABA T., SIAL R., and SHAD S. A. Brain tumor detection: a long short-term memory (LSTM)-based learning model. Neural Computing and Applications, 2020, 32(20): 15965–15973.

CHAHAL P. K., PANDEY S., and GOEL S. A survey on brain tumor detection techniques for MR images. Multimedia Tools and Applications, 2020, 79(29–30): 21771–21814.

SHARIF M., AMIN J., NISAR M. W., ANJUM M. A., MUHAMMAD N., and ALI SHAD S. A unified patch based method for brain tumor detection using features fusion. Cognitive Systems Research, 2020, 59: 273–286.

RAJINIKANTH V., RAJ A.N.J., THANARAJ K. P., and NAIK G. R. A customized VGG19 network with concatenation of deep and handcrafted features for brain tumor detection. Applied Sciences, 2020, 10(10).

WINDISCH P., WEBER P., FÜRWEGER C., EHRET F., KUFELD M., ZWAHLEN D., and MUACEVIC A. Implementation of model explainability for a basic brain tumor detection using convolutional neural networks on MRI slices. Neuroradiology, 2020, 62(11): 1515–1518.

CHANDRAS.K., and BAJPAI M. K. Fractional Crank-Nicolson finite difference method for benign brain tumor detection and segmentation. Biomedical Signal Processing and Control, 2020, 60: 102002.

SHARIF M., AMIN J., RAZA M., YASMIN M., and SATAPATHY S. C. An integrated design of particle swarm optimization (PSO) with fusion of features for detection of brain tumor. Pattern Recognition Letters, 2020, 129: 150–157.

KHAN S. R., SIKANDAR M., ALMOGREN A., UD DIN I., GUERRIERI A., and FORTINO G. IoMT-based computational approach for detecting brain tumor. Future Generation Computer Systems, 2020, 109: 360–367.

TAN W., TIWARI P., PANDEY H. M., MOREIRA C., and JAISWAL A. K. Multimodal medical image fusion algorithm in the era of big data. Neural Computing and Applications, 2020.

HUANG B., YANG F., YIN M., MO X., and ZHONG C. A Review of Multimodal Medical Image Fusion Techniques. Computational and Mathematical Methods in Medicine, 2020: 8279342.

RAJALINGAM B., AL-TURJMAN F., SANTHOSHKUMAR R., and RAJESH M. Intelligent multimodal medical image fusion with deep guided filtering. Multimedia Systems, 2020: 0123456789.

ARIFM., and WANG G. Fast curvelet transform through genetic algorithm for multimodal medical image fusion. Soft Computing, 2020, 24(3): 1815–1836

WANG K., ZHENG M., WEI H., QI G., and LI Y. Multi-modality medical image fusion using convolutional neural network and contrast pyramid. Sensors, 2020, 20(8): 1–17.

YADAV S. P., and YADAV S. Image fusion using hybrid methods in multimodality medical images. Medical and Biological Enginering & Computing, 2020, 58(4): 669–687.

DU J., LI W., and TAN H. Three-layer medical image fusion with tensor-based features. Information Sciences, 2020, 525: 93–108.

WANG Z., CUI Z., and ZHU Y. Multi-modal medical image fusion by Laplacian pyramid and adaptive sparse representation. Computers in Biology and Medicine, 2020, 123: 103823.

MAQSOODS., and JAVED U. Multi-modal Medical Image Fusion based on Two-scale Image Decomposition and Sparse Representation. Biomedical Signal Processing and Control, 2020, 57: 101810.

CHEN J., ZHANG L., LU L., LI Q., HU M., and YANG X. A novel medical image fusion method based on Rolling Guidance Filtering. Internet of Things, 2020, 14: 100172.

DALAL N., and TRIGGS B. Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005, pp. 886–893,

WANG D., WAN J., CHEN J., and ZHANG Q. An online dictionary learning-based compressive data gathering algorithm in wireless sensor networks. Sensors, 2016, 16(10): 1547.

JANYSHABU S. L., & JAYAKUMAR C. Brain Tumor Classification with MRI Brain Images Using 2-Level GLCM Features and Sparse Representation based Segmentation. Proceedings of the Third International Conference on Intelligent Sustainable Systems, 2020.

REFONAA J., & LAKSHMI M. Accurate Prediction of the Rainfall using Convolutional Neural Network and Parameters Optimization using Improved Particle Swarm Optimization. Journal of Advanced Research in Dynamical and Control Systems, 2019, 11(02): 318-328.

DHAMODARAN S., LAKSHMI M. Ensampling data prediction using sparse data in the mobile intelligent system. International Journal of Interactive Mobile Technologies, 2019, 13(10): 106-119.

VIGNESHWARI S., BHARATHI B., SASIKALA T., and MUKKAMALA S. A study on the application of machine learning algorithms using R. Journal of Computational and Theoretical Nanoscience, 2019, 16(8): 3466-3472.

EFRON B., HASTIE T., JOHNSTONE I., and TIBSHIRANI R. Least angle regression. Annals of Statistics, 2004, 32(2): 407–499.


  • There are currently no refbacks.