Report Website

Project Summary

This project revolves around the task of disaster change detection using remote sensing imagery, particularly from satellite sources. The objective is to create a comprehensive dataset and develop models to detect and analyze changes between pre- and post-disaster images. The dataset includes over 60 disaster events, along with pixel-level annotations of disaster-specific changes.

Dataset

The dataset combines multispectral aerial images from Sentinel-2, enriched with auxiliary data such as terrain data from OpenStreetMap (OSM), land cover information from ESA WorldCover, and textual reports from ReliefWeb. The dataset includes high-resolution images with 12 spectral bands, and detailed annotations of disaster impacts were manually added using the CVAT annotation tool.

Data Sources:

  • Sentinel-2 imagery: 10-meter resolution for RGB bands and up to 60 meters for other spectral bands.
  • OpenStreetMap: For contextual geographic information.
  • ESA WorldCover: Provides land cover maps for understanding pre- and post-disaster conditions.
  • ReliefWeb: Disaster-specific textual reports to add context.

Data Processing:

  • Cloud filtering: Only images with less than 10% cloud cover were selected.
  • Temporal window: Images were captured within a 90-day window around the disaster date.
  • Spatial extent: 30x30 km² areas centered around the disaster.

Task Focus

The primary task is change detection, identifying changes in images caused by natural disasters. The dataset is divided into three visibility-based categories:

  1. Visible: Clearly visible disaster changes.
  2. Tiny-visible: Small area changes.
  3. Not visible: Changes not visible in satellite imagery but known through external sources.

Models and Baselines

Several change detection models were tested on this dataset, including:

  1. FC-EF: A single-stream fully convolutional network that merges pre- and post-disaster images.
  2. FC-Siam-Diff: A dual-stream network with a difference operation on extracted features.
  3. FC-Siam-Conc: A dual-stream network that concatenates extracted features from pre- and post-images.
  4. BIT: A transformer-based model.
  5. TinyCD: EfficientNet-based model designed for small-area change detection.

Training involved a small, annotated dataset split into training, validation, and test sets. Models were trained for 20 epochs with a batch size of 1 using Adam optimizer. The evaluation used metrics such as precision, recall, F1 score, IoU, and Overall Accuracy.

Results

  • The FC-EF model showed the best overall performance on early generalization.
  • TinyCD and BIT models struggled with training efficiency and generalization.
  • Visual examples of detected changes across different models demonstrated varying success rates.

Future Work

The next steps for this project include:

  • Expanding the dataset: More disaster types and annotated samples.
  • Leveraging full Sentinel-2 spectral data: Utilize all 12 spectral bands for better analysis.
  • Integrating new tasks: Add tasks such as visual question answering (VQA) to explore the full potential of the multimodal dataset.
  • Model refinement: Explore new architectures and training approaches to improve change detection performance.