VPRS-based regional decision fusion of CNN and MRF classifications for very fine resolution remotely sensed images

Zhang, Ce and Sargent, Isabel and Pan, Xin and Gardiner, Andy and Hare, Jonathon and Atkinson, Peter Michael (2018) VPRS-based regional decision fusion of CNN and MRF classifications for very fine resolution remotely sensed images. IEEE Transactions on Geoscience and Remote Sensing, 56 (8). pp. 4507-4521. ISSN 0196-2892

[thumbnail of FINAL VERSION]
FINAL_VERSION.pdf - Accepted Version
Available under License Creative Commons Attribution-NonCommercial.

Download (1MB)


Recent advances in computer vision and pattern recognition have demonstrated the superiority of deep neural networks using spatial feature representation, such as convolutional neural networks (CNN), for image classification. However, any classifier, regardless of its model structure (deep or shallow), involves prediction uncertainty when classifying spatially and spectrally complicated very fine spatial resolution (VFSR) imagery. We propose here to characterise the uncertainty distribution of CNN classification and integrate it into a regional decision fusion to increase classification accuracy. Specifically, a variable precision rough set (VPRS) model is proposed to quantify the uncertainty within CNN classifications of VFSR imagery, and partition this uncertainty into positive regions (correct classifications) and non-positive regions (uncertain or incorrect classifications). Those “more correct” areas were trusted by the CNN, whereas the uncertain areas were rectified by a Multi-Layer Perceptron (MLP)-based Markov random field (MLP-MRF) classifier to provide crisp and accurate boundary delineation. The proposed MRF-CNN fusion decision strategy exploited the complementary characteristics of the two classifiers based on VPRS uncertainty description and classification integration. The effectiveness of the MRF-CNN method was tested in both urban and rural areas of southern England as well as Semantic Labelling datasets. The MRF-CNN consistently outperformed the benchmark MLP, SVM, MLP-MRF and CNN and the baseline methods. This research provides a regional decision fusion framework within which to gain the advantages of model-based CNN, while overcoming the problem of losing effective resolution and uncertain prediction at object boundaries, which is especially pertinent for complex VFSR image classification.

Item Type:
Journal Article
Journal or Publication Title:
IEEE Transactions on Geoscience and Remote Sensing
Additional Information:
©2018 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Uncontrolled Keywords:
?? electrical and electronic engineeringearth and planetary sciences(all) ??
ID Code:
Deposited By:
Deposited On:
03 Apr 2018 08:34
Last Modified:
14 Jun 2024 01:12