Enhancing Mapping of Precarious Urban Areas with Open Building Footprints Data using a Multimodal Deep Learning Approach
Research project focused on adapting a deep learning model to combine Sentinel satellite imagery with open building footprints data.
Structure (in short)
1. Context
The project was done in partnership with the United Nations Innovation Technology Accelerator for Cities (UNITAC), who provided ground-truth data and local expertise on precarious urban areas in Central America. The collaboration aimed to enhance precarious area mapping capabilities in Central American cities.
2. Research Topic
The idea for the project is to use a pretrained model from the DeepLNAfrica project, trained for a similar task on African cities, and adapt its architecture to accommodate the detailed information of building footprints alongside Sentinel imagery to improve its predictive ability.
3. Methods
The pretrained Deeplabv3 architecture was adapted with an additional encoder, fusing Sentinel and building footprints features at multiple stages, and introducing a lateral connection between the encoder and decoder.
Above is the final adapted model architecture fusing Sentinel imagery with higher-resolution building footprints raster, visualised in LaTeX using PlotNeuralNet.
4. Results
The adapted multimodel model showed a 30% improvement in predictive accuracy as measured by the F1 score compared to the original Sentinel-only model.
Model predictions on an unseen city.
Paper Abstract
Improving the living standards for urban residents in informal settlements is a critical objective of global development, aligning with the United Nations Sustainable Development Goals (SDGs), particularly SDG 11. Despite significant efforts by international organisations and researchers, mapping and defining these areas remains challenging due to their inherently uncertain and variable characteristics. Advances in Earth Observation (EO) data and machine learning, particularly convolutional neural networks (CNNs), have enhanced the ability to identify and monitor these areas. However, open satellite imagery is often insufficient for mapping informal settlements, as its resolution lacks the detail needed to capture their unique urban morphology.
This research explores the potential of integrating open building footprint data from the Overture Maps Foundation with optical imagery from Sentinel-2 to improve these efforts. The study utilises data on precarious areas in Central American capital cities, collected by the United Nations Innovation Technology Accelerator for Cities (UNITAC) and employs a multimodal deep learning model, building on a pretrained DeeplabV3 model developed for the DeepLNAfrica project (Büttner et al. 2024).
The enhanced model demonstrated a nearly 30% improvement in predictive accuracy as measured by the F1 score compared to the original Sentinel-only model. Furthermore, this project estimates the proportion of the population living in precarious areas across 24 major cities in the region, showcasing the potential of the developed framework for tracking progress towards SDG 11. While challenges persist in defining and mapping these inherently variable areas, our approach offers a scalable, open-data solution that significantly improves upon existing methods.