Building a Global Multimodal Earth Observation Dataset by Integrating Mapillary Street-Level Imagery with Sentinel-1/2 Satellite Data
Earth observation (EO) has become a fundamental tool for monitoring environmental processes, land use dynamics, and water systems at regional and global scales. Satellites such as the Sentinel-1 and Sentinel-2 provide high-quality data with global coverage and frequent revisit times. Despite their strengths, observations from satellites often lack fine-scale semantic information about objects and structures visible at ground level.
Street-level imagery platforms such as Mapillary provide billions of geotagged images collected worldwide through crowdsourced contributions. These images contain detailed information about roads, drainage infrastructure, coastal protection structures, vegetation, riverbanks, urban waterways, and flood-prone environments. Such ground observations complement satellite data by providing detailed contextual information about landscapes and infrastructure interacting with water systems.
Integrating satellite and street-level imagery therefore presents a unique opportunity to develop multimodal environmental datasets that combine large-scale Earth observation with detailed ground-level perspectives. Despite the availability of large satellite archives and massive collections of geotagged street imagery, there is currently no global dataset systematically linking Sentinel-1, Sentinel-2, and street-level imagery.
This thesis aims to address this gap by developing a global multimodal dataset that integrates Mapillary street-level images with Sentinel-1 and Sentinel-2 satellite observations, with a particular focus on environmental and water-related applications relevant to hydrological and coastal research.
Visit DHI