Add example notebooks

[AdeelH]

As a follow-up to #1395, we should add Python notebooks that demonstrate how to use Raster Vision as a library.

If feasible, it would also be nice to automatically add rendered versions of these notebooks to the docs. See: https://docs.readthedocs.io/en/stable/guides/jupyter.html.

Some ideas:

Basics

• Reading raster data (Add example notebooks + multiple major changes #1470)
  • RasterSource, MultiRasterSource, CRSTransformer, RasterTransformer
• Reading vector data (Add example notebooks + multiple major changes #1470)
  • VectorSource, VectorTransformer, RasterizedSource
• Preparing labels (Add example notebooks + multiple major changes #1470)
  • Labels, LabelSource
• Scenes and AoIs (Add more notebooks + misc. doc fixes and improvements #1506)
  • Scene
• GeoDatasets (Add example notebooks + multiple major changes #1470)
  • GeoDataset, SlidingWindowGeoDataset, RandomWindowGeoDataset
  • different ways of instantiating
• Training (Add more notebooks + misc. doc fixes and improvements #1506)
  • data augmentation
  • tensorboard
• Making and saving predictions (Add more notebooks + misc. doc fixes and improvements #1506)
  • Learner.predict_dataset(), LabelStore
• Evaluating predictions (Add more notebooks + misc. doc fixes and improvements #1506)
  • Evaluator
• Visualizer (Lewis) (Factor out dataset visualization #1476)

Other

• Full workflow for Potsdam segmentation that matches examples repo (Lewis)
• Full workflow for Potsdam segmentation using RV + PyTorch Lightning (Lewis) (Add RV + Lightning notebook #1546)
• Reading GroundWork exports / reading STAC catalogs
• Training on pre-chipped/non-georeferenced data (Add a tutorial notebook for reading pre-chipped datasets #1602)

[lewfish]

Are you thinking that there will be different notebooks for the different checkbox points you listed? I was thinking we would have notebooks that covered whole workflows for specific datasets/tasks that would have the same scope as the examples on the Examples page that we already have, and in the course of these we would cover all these topics.

I was thinking that we might want to add an example of a lower-resolution land cover dataset since that's a common use case and isn't covered by the existing example which are all about cars and buildings and high-res imagery.

[AdeelH]

Are you thinking that there will be different notebooks for the different checkbox points you listed? I was thinking we would have notebooks that covered whole workflows for specific datasets/tasks

Yes, I'm hoping to have both. Shorter ones are easier to look up info in. I'm happy to take those on.

Also, this is a first draft. I think some of these can be merged together.

[AdeelH]

Please feel free to edit the issue body.

[AdeelH]

An example of RV-as-library code can be seen here: https://github.com/raster-foundry/raster-foundry/tree/develop/app-hitl/hitl/src/hitl/rv

[AdeelH]

I was thinking that we might want to add an example of a lower-resolution land cover dataset since that's a common use case

@lewfish, do you know of a convenient way to get this kind of data (ideally from within a notebook)?

[lewfish]

There are some datasets like this on MLHub that we could access through STAC such as https://mlhub.earth/data/microsoft_chesapeake

[lewfish]

I'd like to show how you can take an off-the-shelf PyTorch model (perhaps from TorchGeo) and then use RV to make predictions on Scene objects. Is that something that is supported by the recent refactoring?

[AdeelH]

Yes, but it's tied up in the Learner as well. You need to create a Learner with Learner(model=your_model, training=False) and then do basically this: https://github.com/raster-foundry/raster-foundry/blob/develop/app-hitl/hitl/src/hitl/rv/predict.py

[lewfish]

Yes, but it's tied up in the Learner as well. You need to create a Learner with Learner(model=your_model, training=False) and then do basically this: https://github.com/raster-foundry/raster-foundry/blob/develop/app-hitl/hitl/src/hitl/rv/predict.py

Hmm, ok. I'll think about potentially simplifying this.

[ammarsdc]

Hi @lewfish @AdeelH. I would like to share our latest implementation for reading GroundWork STAC that works for us. Maybe it might somehow help to prepare the STAC reading part.

from datetime import datetime as dt
from os.path import join, dirname
from pathlib import Path

from rastervision.core.rv_pipeline import *
from rastervision.core.data import *
from rastervision.pytorch_backend import *
from rastervision.pytorch_learner import *
from rastervision.core.utils.stac import read_stac


def get_config(
    runner,
    ortho_tif_uri: str,
    aoi_uri: str,
    stac_zip_uri: str,
    output_uri: str,
    auto_output_uri: bool = True,
    num_epochs: int = 30,
    num_ds: int = 10,
    model_name: str = "resnet50",
    test_mode: bool = False,
) -> SemanticSegmentationConfig:

    chip_sz = 300
    img_sz = chip_sz

    class_config = ClassConfig(
        names=["object", "background"], colors=["blue", "white"], null_class="background"
    )

    # AOI Setup is here
    def make_scene(id: str, stac_item: dict) -> SceneConfig:
        label_store = SemanticSegmentationLabelStoreConfig(
            rgb=True, vector_output=[PolygonVectorOutputConfig(class_id=0)]
        )

        rasterizer_config = RasterizerConfig(background_class_id=1)

        vector_source = GeoJSONVectorSourceConfig(
            uri=stac_item["label_uri"],
            ignore_crs_field=True,
            transformers=[ClassInferenceTransformerConfig(default_class_id=0)],
        )

        raster_source = RasterizedSourceConfig(
            vector_source=vector_source,
            rasterizer_config=rasterizer_config,
        )

        label_source = SemanticSegmentationLabelSourceConfig(
            raster_source=raster_source
        )

        raster_source = RasterioSourceConfig(
            channel_order=[0, 1, 2], uris=[ortho_tif_uri]
        )

        return SceneConfig(
            id=id,
            raster_source=raster_source,
            label_source=label_source,
            label_store=label_store,
            aoi_uris=[aoi_uri]
            # aoi_geometries=aoi_geometries
        )

    # STAC reading and processing are here
    stac_dir = dirname(stac_zip_uri)
    stac_unzip_dir = join(stac_dir, "stac")
    stac_items = read_stac(stac_zip_uri, stac_unzip_dir)

    scenes = [
        make_scene(f"scene_{i}", stac_item) for i, stac_item in enumerate(stac_items)
    ]

    dataset = DatasetConfig(
        class_config=class_config, train_scenes=scenes, validation_scenes=scenes
    )

    chip_options = SemanticSegmentationChipOptions(
        window_method=SemanticSegmentationWindowMethod.sliding, stride=chip_sz
    )

    # For larger amount of training and validate image, change here at `max_windows``
    window_opts = GeoDataWindowConfig(
        method=GeoDataWindowMethod.random,
        size=chip_sz,
        size_lims=(chip_sz, chip_sz + 1),
    )

    data = SemanticSegmentationGeoDataConfig(
        img_sz=img_sz, scene_dataset=dataset, window_opts=window_opts
    )

    solver = SolverConfig(
        num_epochs=num_epochs, test_num_epochs=num_epochs, one_cycle=True
    )

    backend = PyTorchSemanticSegmentationConfig(
        model=SemanticSegmentationModelConfig(backbone=Backbone[model_name.lower()]),
        solver=solver,
        data=data,
        test_mode=test_mode,
    )

    datetime_dir_name = dt.now().strftime("%Y%m%d_%H%M%S_%f")
    pipeline_dir_name = datetime_dir_name
    aoi_dir_name = Path(aoi_uri).stem

    output_dir_name = (
        join(
            f"mode-{'test' if test_mode else 'full'}",
            f"epochs-{num_epochs}",
            f"dataset-{num_ds}",
            f"model-{model_name}",
            f"aoi-{aoi_dir_name}",
            f"{pipeline_dir_name}",
        )
        if auto_output_uri
        else ""
    )

    output_uri = join(output_uri, output_dir_name)

    # Specifies the details about how the commands within the pipeline will execute
    # ie. which files, what methods, what hyperparameters, etc.
    # https://docs.rastervision.io/en/latest/pipelines.html#configuring-rvpipelines
    pipeline = SemanticSegmentationConfig(
        root_uri=output_uri,
        dataset=dataset,
        backend=backend,
        train_chip_sz=chip_sz,
        predict_chip_sz=chip_sz,
    )

    return pipeline