Skip to content
Branch: master
Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information. Use fastai plugin, add levels, improve plots Jul 31, 2019


The purpose of this experiment set is to measure how errors in training labels affect accuracy in the learned model. This uses the Spacenet Vegas building semantic segmentation dataset, based on the Vegas example.

For each noise type (ie. randomly shifting and deleting buildings) and noise level, we create a noisy label set based on the original ground truth labels. For each noisy label set, we train a model and validate on the original, uncorrupted labels.


  • Running this requires To run this, place this repo directory within the examples directory, and follow instructions in the fastai plugin for running an example.
  • Run this inside the container: export PYTHONPATH=/opt/src/examples/raster-vision-experiments/:"$PYTHONPATH"
  • Download the data following instructions in the Vegas example.
  • Set paths to data and RV output by modifying the constants at the start of
  • Generate the noisy labels by running python -m noisy_buildings_semseg.prep
  • Sync noisy labels to cloud using aws cli.
  • Run a small test local experiment using rastervision -p fastai run local -e noisy_buildings_semseg.exp -a test True -a use_remote_data False
  • Run a full remote experiment using rastervision -p fastai run aws_batch -e noisy_buildings_semseg.exp -a test False -a use_remote_data True --splits 4
  • When the individual experiments finish running, plot some curves based on the evaluations using python -m noisy_buildings_semseg.plot_separate_curves and python -m noisy_buildings_semseg.plot_separate_curves, and images of noisy labels and predictions using python -m noisy_buildings_semseg.plot_images.
You can’t perform that action at this time.