Aerial Imagery Workflows

Aerial workflows start from a georeferenced raster (a drone orthomosaic you already have, or a basemap rAPIdtools stitches from Bing or Google tiles), discover or load assets, optionally clean their geometry, crop one image per asset, and hand the crops to a vision-language model.

Overview

Step	Stage	Component	Output
Get a raster	-	BingOrthomosaicExtractor, GoogleOrthomosaicExtractor	GeoTIFF in EPSG:4326
Find assets	DETECT	SAM3OrthoFeatureExtractor	PhysicalAssetCollection of WGS84 polygons
Clean geometry	REGULARIZE	BuildingRegularizer, RoadwayRegularizer	footprints or centerlines
Crop per asset	EXTRACT_IMAGERY	AerialImageryExtractor	one ImageAsset per asset
Describe	ANALYZE	AssetAnalyzer(load(...))	attributes on each asset

Every component is a callable that takes a collection and returns one, so you can chain them by hand or put them in a Pipeline, which sorts them by stage (see Core Concepts). API details for each class are on the documentation site.

Step 1: get a raster

Rasters can be in any CRS; OrthomosaicReader reprojects on read. If you only have footprints, stitch a basemap without an API key:

from rapidtools import BingOrthomosaicExtractor, BoundingBox

region = BoundingBox.from_raster('eaton_patch_20250214.tiff')   # or BoundingBox(min_x, min_y, max_x, max_y)
tiff = BingOrthomosaicExtractor(zoom_level=19, max_workers=10)(region, output_path='eaton_bing.tiff')

GoogleOrthomosaicExtractor is a drop-in alternative that goes to about zoom 21 in cities and offers a hybrid layer with labels (layer='y'). These are the unofficial endpoints of the Google Maps web app; failures are logged and the tile is skipped. For a quick set of training chips with no raster in between, GoogleAerialImageExtractor and BingAerialImageExtractor crop tiles per polygon with process_geojson(path, buffer_percent=0.1, pad_to_square=True, resize_to=(640, 640)).

Step 2: find assets with SAM 3

SAM3OrthoFeatureExtractor tiles the raster into square ground patches, runs SAM 3 with your text prompt on each batch, and vectorizes the masks to WGS84 polygons.

from rapidtools import SAM3OrthoFeatureExtractor

buildings = SAM3OrthoFeatureExtractor(
    prompt='building roof',
    patch_size=50, unit='meters',   # tile size on the ground
    overlap_ratio=0.25,             # avoid cutting objects at tile edges
    batch_size=4,
    threshold=0.5, mask_threshold=0.4,
)('eaton_bing.tiff')
buildings.to_geojson('buildings_preliminary.geojson')

Parameter	Default	Notes
patch_size, unit	30.0, 'feet'	'pixels', 'meters', 'feet', 'yards', 'kilometers', 'miles'. Small patches keep texture for small objects; 100 m suits roads and vegetation.
overlap_ratio	0.15	Fraction of overlap between neighbouring tiles.
threshold, mask_threshold	0.5, 0.5	Detection confidence and mask binarization. Lower both (0.1 to 0.25) for faint classes such as vegetation or asphalt.
merge_overlaps	True	Dissolve touching detections (roofs, roads). Set False for countable objects such as vehicles: one asset per instance with a confidence attribute.
max_missing_data_ratio	0.95	Tiles emptier than this are skipped.
load_in_4bit, device	True, 'auto'	Memory and placement of the SAM 3 weights.
raster_path	None	Set it to use the extractor as a Pipeline step instead of calling it with a path.

Step 3: regularize geometry

Masks from a basemap are jagged, sometimes merged with a neighbour, and occasionally hallucinated. BuildingRegularizer re-runs SAM 3 on higher-resolution crops around each seed and keeps a mask only when it agrees with the seed (IoU of at least min_overlap_ratio, or recall). Merged seeds are split, gaps shorter than max_gap_bridge_ft are bridged, and dropped seeds get a lower-threshold retry pass. It needs crops with georeferencing, which AerialImageryExtractor writes:

from rapidtools import AerialImageryExtractor, BuildingRegularizer

with_crops = AerialImageryExtractor(
    'eaton_bing.tiff', save_directory='output/crops',
    buffer_asset='20 m', force_square_image=True,
)(buildings)
final = BuildingRegularizer(prompt='building roof', batch_size=8)(with_crops)
final.to_geojson('buildings_final.geojson')

RoadwayRegularizer turns raw road polygons into a Voronoi skeleton, heals and merges the graph, samples widths and drops stubs shorter than min_network_length_ft:

from rapidtools import RoadwayRegularizer, SAM3OrthoFeatureExtractor

raw = SAM3OrthoFeatureExtractor(prompt='paved road, street', patch_size=100, unit='meters',
                                threshold=0.25, mask_threshold=0.25)('eaton_patch_20250214.tiff')
centerlines, polygons = RoadwayRegularizer(min_width_ft=22)(raw)
centerlines[0].attributes      # {'asset_type': 'road_centerline', 'width_ft': ..., 'azimuth_deg': ...}

Pass output='centerlines' or output='polygons' to get a single collection, which is what makes the regularizer usable as a Pipeline step.

Step 4: crop one image per asset

AerialImageryExtractor crops the raster around each asset, saves a JPEG and attaches an ImageAsset with the ID <asset_id>_<image_prefix> and three properties: wgs84_bounds, native_georef and source_raster. dataset may be one TIFF, a folder of TIFFs (searched recursively) or a list; each raster is opened once and only the assets inside its extent are processed.

Parameter	Default	Notes
buffer_asset	'20%'	Margin around the footprint: a percentage, a distance ('10 m', '30 ft') or a bare number in raster units. buffer_m=10 is a metric shortcut.
force_square_image	True	Every crop covers a square area on the ground.
max_missing_data_ratio	0.2	Skip crops with more nodata than this (judged from the raster mask, alpha band or nodata value).
min_footprint_coverage	None	Instead keep an asset if this fraction of its footprint is imaged, for assets on the flight edge.
pad_edges	None	Pad edge crops with nodata to full size; defaults to True when min_footprint_coverage is set.
image_prefix	'aerial'	Used in file names and image IDs; pick a different prefix per survey.
keep_multiple_copies	False	Append a counter instead of overwriting when rasters overlap.
cancel_event	None	threading.Event checked between assets.

Outlining the target asset

In dense neighbourhoods a model often describes the wrong building. overlay_asset_outline=True draws the asset on the crop so the prompt can say "the outlined building".

outline_shape	What is drawn
'geometry' (default)	The asset's own geometry; a dot for point assets.
'bbox'	Axis-aligned bounding box.
'rotated_bbox'	Minimum rotated rectangle.
'convex_hull'	Convex hull.
'corners'	L-shaped brackets at the corners of the rotated rectangle, leaving the roof edge unobstructed.

outline_buffer pushes the outline away from the asset (pixels, '10%' of the asset's pixel extent, or '2 m' converted with the crop's ground sampling distance; a buffered point becomes a ring). outline_width is pixels or '1%' of the shorter image side; outline_color takes any PIL colour. Corner brackets are a good default for damage assessment.

Step 5: analyze the crops

from rapidtools import AerialImageryExtractor, AssetAnalyzer, PhysicalAssetCollection, Pipeline, download_dataset
from rapidtools.models import GenerationConfig, load

raster_path, footprint_path, prompt_path = download_dataset(
    ['eaton_patch2', 'altadena_sample_buildings', 'aerial_chs_prompts']
)
buildings = PhysicalAssetCollection.from_geojson(footprint_path)

pipeline = Pipeline([
    AerialImageryExtractor(
        dataset=raster_path, save_directory='output/crops',
        overlay_asset_outline=True, outline_shape='corners', outline_buffer='2 m',
        image_prefix='eaton_feb25',
    ),
    AssetAnalyzer(
        load('gemma4', model_id='google/gemma-4-E2B-it'),   # fits an 8 GB GPU
        prompt=prompt_path,
        batch_size=8,
        generation=GenerationConfig(json_mode=True, temperature=0.0),
    ),
])
result = pipeline.run(buildings).filter_empty()
result.to_geojson('eaton_footprints_CHS.geojson', ignore_properties=['image_assets'])

Answers land in attributes prefixed with the provider key (gemma4_chs_level); see Vision-Language Models for providers, prompts and rate limits, and Data Formats and Export for what ends up in the file.

Reading rasters directly

OrthomosaicReader is the reader behind the extractors. It opens a GeoTIFF once, reports its WGS84 extent, crops a patch around any geometry with the same buffering rules, and slides a window over the raster while skipping empty tiles:

from rapidtools import OrthomosaicReader
from rapidtools.processing.vectorize import mask_to_wgs84_polygons

with OrthomosaicReader('ortho.tif') as reader:
    bbox = reader.get_mosaic_bbox_wgs84()
    for image, bounds, georef in reader.generate_tiles(patch_size=50, unit='meters', return_georef=True):
        mask = ...                                  # 2D array from your own model
        polygons = mask_to_wgs84_polygons(mask, georef=georef)

PatchGeoref holds the patch's affine transform and CRS. Vectorizing in that grid and reprojecting is exact for projected and rotated rasters; mapping pixels through wgs84_bounds alone is exact only for north-up WGS84 rasters. AerialImageryExtractor stores the same record as the native_georef image property, so BuildingRegularizer reads it back automatically.

Common pitfalls

• Assets on the flight edge disappear. The default max_missing_data_ratio=0.2 drops them. Use min_footprint_coverage=0.3 with pad_edges=True.
• One building, several detections. Overlapping tiles see the same roof twice; merge_overlaps=True (default) dissolves them. Keep it False only for countable objects.
• Cars parked side by side become one asset. Dissolving fuses touching objects; detect vehicles with merge_overlaps=False.
• Nothing detected for faint classes. Lower threshold and mask_threshold (the vegetation example uses 0.1) and shrink patch_size.
• Skewed polygons on UTM or rotated rasters. Crops from older runs carry only wgs84_bounds; re-run the extractor so native_georef is present.
• GPU out of memory. Lower batch_size, keep load_in_4bit=True, and do not hold two local models at once.
• Basemap looks old or missing. Try the other provider; Bing and Google refresh different areas at different times.
• Second run overwrites crops. Change image_prefix or set keep_multiple_copies=True.
• Long jobs you cannot stop. Pass one cancel_event to the pipeline and every step; partial results stay on the assets (OperationCancelled).

See also Street-Level Imagery Workflows, Examples and Troubleshooting and FAQ.