M 2 -STF : Integration of Multi-modal Data for Spatio-temporal Fusion

Wang, Qunming and Li, Aijing and Tang, Yijie and Atkinson, Peter M. (2025) M 2 -STF : Integration of Multi-modal Data for Spatio-temporal Fusion. IEEE Transactions on Geoscience and Remote Sensing, 63: 5408216. pp. 1-16. ISSN 0196-2892

Text (M2-STF Wang et al 2025)
M2-STF_Wang_et_al_2025.pdf - Accepted Version
Available under License Creative Commons Attribution.
Download (2MB)

Abstract

Spatio-temporal fusion is a general technique used to blend fine spatial resolution and fine temporal resolution remote sensing data from multiple sensors, to generate time-series data with both fine spatial and temporal resolutions. It has received increasing attention in recent years. Drastic changes in land surface, however, pose great challenges for spatio-temporal fusion. To address this issue, this paper proposed a spatio-temporal fusion method which integrates multi-modal data (M2-STF), specifically SAR data with optical data. Considering the scenario of flooding (which causes drastic land surface changes) as an example, this study focused on spatio-temporal fusion based on Sentinel-2 MSI and Sentinel-3 OLCI data, and developed the M2-STF method by integrating Sentinel-1 SAR data. For the changed area, M2-STF integrates the Sentinel-2 image at the known time and the Sentinel-1 SAR image at the prediction time to obtain a more accurate fine spatial resolution classification map at the prediction time. Based on this map, a spatial unmixing model and spatial interpolation model were developed taking into account both homogeneity and heterogeneity characteristics, which were then combined into a homogeneity index. For the unchanged area, a new similar pixel selection strategy was constructed to exclude the influence of similar pixels from the changed area. In the experiments, three regions were selected for validation, and M2-STF was compared with five typical spatio-temporal fusion methods. By integrating Sentinel-1 SAR data at the prediction time, the accuracy of spatio-temporal fusion was increased remarkably, especially when the land surface changes greatly from the known to the prediction times. Specifically, the M2-STF method outperforms all five benchmark methods, by reducing root mean square error (RMSE) by at least 16%.