Single Satellite Imagery Simultaneous Super-resolution and Colorization using Multi-task Deep Neural Networks

Liu, Heng and Fu, Zilin and Han, Jungong and Shao, Ling and Liu, Hongshen (2018) Single Satellite Imagery Simultaneous Super-resolution and Colorization using Multi-task Deep Neural Networks. Journal of Visual Communication and Image Representation, 53. pp. 20-30. ISSN 1047-3203

[thumbnail of 1-s2.0-S1047320318300488-main]
Preview
PDF (1-s2.0-S1047320318300488-main)
1_s2.0_S1047320318300488_main.pdf - Accepted Version
Available under License Creative Commons Attribution-NonCommercial-NoDerivs.

Download (7MB)

Abstract

Satellite imagery is a kind of typical remote sensing data, which holds preponderance in large area imaging and strong macro integrity. However, for most commercial space usages, such as virtual display of urban traffic flow, virtual interaction of environmental resources, one drawback of satellite imagery is its low spatial resolution, failing to provide the clear image details. Moreover, in recent years, synthesizing the color for grayscale satellite imagery or recovering the original color of camouflage sensitive regions becomes an urgent requirement for large spatial objects virtual reality interaction. In this work, unlike existing works which solve these two problems separately, we focus on achieving image super-resolution (SR) and image colorization synchronously. Based on multi-task learning, we provide a novel deep neural network model to fulfill single satellite imagery SR and colorization simultaneously. By feeding back the color feature representations into the SR network and jointly optimizing such two tasks, our deep model successfully achieves the mutual cooperation between imagery reconstruction and image colorization. To avoid color bias, we not only adopt the non-satellite imagery to enrich the color diversity of satellite image, but also recalculate the prior color distribution and the valid color range based on the mixed data. We evaluate the proposed model on satellite images from different data sets, such as RSSCN7 and AID. Both the evaluations and comparisons reveal that the proposed multi-task deep learning approach is superior to the state-of-the-art methods, where image SR and colorization can be accomplished simultaneously and efficiently.

Item Type:
Journal Article
Journal or Publication Title:
Journal of Visual Communication and Image Representation
Additional Information:
This is the author’s version of a work that was accepted for publication in Journal of Visual Communication and Image Representation Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Journal of Visual Communication and Image Representation, 53, 2018 DOI: 10.1016/j.jvcir.2018.02.016
Uncontrolled Keywords:
/dk/atira/pure/subjectarea/asjc/2200/2208
Subjects:
?? IMAGE SUPER-RESOLUTIONSATELLITE IMAGE COLORIZATIONDEEP NEURAL NETWORKSMULTI-TASK LEARNINGMEDIA TECHNOLOGYSIGNAL PROCESSINGCOMPUTER VISION AND PATTERN RECOGNITIONELECTRICAL AND ELECTRONIC ENGINEERING ??
ID Code:
123887
Deposited By:
Deposited On:
07 Mar 2018 11:54
Refereed?:
Yes
Published?:
Published
Last Modified:
17 Sep 2023 02:14