Street Level Imagery Workflows

Street-level views show facades, debris and recovery work that aerial imagery cannot; rAPIdtools fetches them from Google Street View without a key or from Mapillary with a free token, keeps only the views that actually show the asset, and can also discover objects along a survey from Mapillary's own detections.

Two starting points

You have	Use	Stage
An inventory (footprints, poles, points)	GoogleStreetViewImageExtractor or MapillaryImageExtractor	EXTRACT_IMAGERY
Only a survey area	MapillaryFeatureExtractor then MapillaryObjectImageExtractor	DETECT, then EXTRACT_IMAGERY

Source	Key needed	Notes
Google Street View	No	Public endpoints of the Google Maps web app; availability may change.
Mapillary	Free access token (`'MLY	...'`)
Bing Streetside	Bing Maps key	Not supported: the bubble lookup rejects unauthenticated requests.

The examples download a shared token with download_dataset('mapillary_token'). The token is redacted from logged tile and Graph API errors. Full signatures are on the documentation site.

Google Street View

The extractor finds the nearest official panorama within search_radius_m of each asset centroid, downloads its tiles, and crops the horizontal span that covers the footprint.

from rapidtools import GoogleStreetViewImageExtractor, PhysicalAssetCollection

buildings = PhysicalAssetCollection.from_geojson('buildings.geojson')
street = GoogleStreetViewImageExtractor(
    'output/streetview',
    search_radius_m=50,
    max_images_per_asset=2,     # extra viewpoints from linked panoramas
    vertical_crop=(0.2, 0.9),   # trim sky and the camera car
    save_depth_map=True,        # decoded depth map as .npy
)
buildings = street(buildings)
buildings['bldg_01'].image_assets[0].properties['pano_id']

Parameter	Default	Notes
search_radius_m	50.0	Raise it on rural roads.
zoom	3	Panorama tile zoom 0 to 5 (zoom 3 is 4096 x 2048 px).
max_images_per_asset	1	Further views come from the panorama's neighbour links, nearest first.
fov_buffer_deg, min_fov_deg	10.0, 30.0	Margin on both sides of the footprint; floor so distant assets do not become slivers.
vertical_crop	(0.0, 1.0)	Fractions of the height to keep.
save_panorama, save_depth_map	False, False	Also write the stitched panorama and the depth map.
image_prefix, max_workers	'gsv', 5	Assets are processed concurrently.

Each crop's properties include pano_id, camera_latitude, camera_longitude, camera_heading, camera_distance_m, capture_date, view_angle_start, view_angle_end, plus panorama_path and depth_map_path when saved.

Mapillary panoramas of known assets

MapillaryImageExtractor fetches regional image metadata once, indexes it spatially, and casts rays from each candidate camera to the asset to reject views blocked by a neighbour. It downloads one panorama per viewing direction: the four principal axes of the footprint, plus the corners when cast_corner_rays=True.

from rapidtools import MapillaryImageExtractor

extractor = MapillaryImageExtractor(
    access_token='MLY|...',
    save_directory='output/street_feb',
    start_date='2025-01-01', end_date='2025-05-01',   # capture-date window, YYYY-MM-DD
    filter_rapid_only=True,
    cast_corner_rays=False,
    smart_crop=True,                                  # remove sky and the collection vehicle
    max_images_per_asset=8,
)
buildings = extractor(buildings)
buildings['bldg_01'].image_assets[0].properties['view_angle']   # 'major_axis_0'

Parameter	Default	Notes
start_date, end_date	''	Inclusive bounds; run once per survey into separate folders for longitudinal studies.
filter_rapid_only	True	False searches all public uploads.
cast_corner_rays	False	Up to 8 views instead of 4.
smart_crop	True	Uses the semantic mask to trim sky and vehicle.
strict_content_filter	False	Read the segmentation first and skip images where the asset type is not visible, saving the JPEG download. Needs asset_type_mapping or label_mapper.
asset_type_mapping	None	e.g. {'building': ['construction--structure--building']}.
label_mapper	None	A MapillaryLabelMapper(load('gemma4')) that translates plain English to Mapillary labels.
image_prefix, max_images_per_asset, max_workers	'street', 8, 10	Major-axis views are kept first.

Crops carry view_angle and original_pano_id. To analyze one survey at a time, give the analyzer an image_filter, for example lambda img: 'street_feb' in str(img.path); the recovery example does this twice with the same loaded model.

Discovering objects along a survey

When the question is "where are all the vehicles or poles along this survey", MapillaryFeatureExtractor builds the inventory from the imagery. Mapillary segments every image it hosts, so detections are read as metadata with no pixel downloads.

import rapidtools as rt
from rapidtools.models import GenerationConfig, load

region = rt.BoundingBox(min_x=-117.495, min_y=47.708, max_x=-117.482, max_y=47.717)  # 1 km of Spokane
token = 'MLY|...'

pipeline = rt.Pipeline([
    rt.MapillaryFeatureExtractor(
        classes=['vehicles'],            # plain English or exact Mapillary labels
        access_token=token, region=region,
        start_date='2026-08-01', end_date='2026-08-31',   # one survey window
        frame_spacing_m=3.0,             # one frame every 3 m is enough to triangulate
        min_observations=3,              # seen in at least three frames
        camera_height_m=2.4,             # roof-mounted rig on the RAPID vehicle
        reid=True,                       # merge look-alikes from repeated passes
    ),
    rt.MapillaryObjectImageExtractor(
        save_directory='output/crops', access_token=token,
        max_images_per_asset=2, image_size='2048', crop_buffer='40%',   # the two widest views
        overlay_asset_outline=True, outline_shape='rotated_bbox', outline_color='red',
    ),
    rt.DetectionVerifier(                # drops what a model says is not a vehicle
        load('gemini', model_id='gemini-3.5-flash-lite'),   # reads GOOGLE_API_KEY
        min_visible_fraction=0.5,
    ),
    rt.DuplicateResolver(                # merges pairs the model is sure are one vehicle
        load('gemini', model_id='gemini-3.5-flash-lite'),
        max_distance_m=8.0, min_confidence=0.9,
    ),
    rt.AssetAnalyzer(
        load('gemini', model_id='gemini-3.8-flash'),
        prompt='Return JSON with "condition": intact, damaged, debris or unclear.',
        max_workers=4, generation=GenerationConfig(json_mode=True, temperature=0.0),
    ),
])
vehicles = pipeline.run(rt.PhysicalAssetCollection())
vehicles[0].attributes['localization']       # 'triangulated' or 'single_view'
vehicles.to_geojson('vehicles.geojson', ignore_properties=['image_assets'])

How it works:

1. Listing and clipping. Mapillary's coverage tiles are about 2 km wide, so a listing for a small box returns far more frames than the box holds. The extractor keeps only cameras inside region and the date window, then thins each sequence to one frame per frame_spacing_m. If you call MapillaryClient.fetch_images_in_bbox yourself, clip with ImageCollection.filter_by_bbox and thin with rapidtools.processing.street_localization.thin_frames.
2. Ego-vehicle filter. The camera car sits at the same place in every frame of a sequence while a parked car drifts across the frame. Boxes that recur in at least ego_min_recurrence of a sequence's frames are dropped before anything else (estimate_ego_mask).
3. Each sighting becomes a ray. The polygon's centre column and Mapillary's full camera pose give a bearing; its lowest point and camera_height_m give a ground-contact range, which is only a prior. Nested duplicate outlines and the two halves of a panorama seam are dropped frame by frame.
4. Tracking and triangulation. Within each sequence, detections are linked frame to frame by bearing continuity and each track is triangulated robustly from its rays. A track whose rays disagree, or whose ground ranges contradict the intersection, was a moving vehicle and is dropped, as is anything on the line the camera drove. Tracks and single views are then merged with a covariance-aware gate, so a car seen from two passes becomes one object while two cars seen in the same frame never do. Objects carry localization ('triangulated' or 'single_view'), position_sigma_m, parallax_deg and n_images.
5. Pieces of a neighbour are folded in. Where vehicles line up along the line of sight, a driveway seen end-on, the tracker can slide from one to the next, and a detector can cut one car in two past a pole. The second object that leaves is judged on what its sightings show: when its closest sightings point at a better-located neighbour, at its range, it is folded in. An object never seen within 20 m is not trusted on its own whatever its parallax, because ground ranges read from that far are off by a third; it is judged by its closest sightings alone and joins the close pass they show, with an end piece of a long vehicle allowed half a vehicle off centre. On by default (merge_pieces_of_neighbours).
6. Only what was seen properly is reported. A weak object never seen within close_range_m (20 m) whose closest rays point at a vehicle already placed from a close pass is a far sighting of that vehicle and is dropped; one pointing elsewhere, a car in a driveway behind the kerb, stays. An object whose outline never reached min_sighting_deg (6 degrees, 34 px on the 2048 thumbnail) was never more than a speck and is not reported, and single views are not reported unless report_single_views=True. On a labelled block these three rules took precision from 52 % to 79 % at 97 % recall.
7. Repeated passes are reconciled by appearance. With reid=True, objects from different sequences within reid_max_distance_m are compared with the image tower of CLIP ViT-B/16 on one 2048-pixel crop each and merged above a threshold picked per backbone. Only objects with a sighting at least reid_min_width_px wide (100 px at 2048) are compared: smaller crops score alike whether or not they show the same vehicle. Appearance only ever adds merges.
8. Verification. DetectionVerifier (stage VERIFY, between cropping and analysis) shows each object's crops to a model and removes what is not the requested class or is less than min_visible_fraction visible. Give it the widest views (the cropper's default view_selection='widest'): a close view can be an end-on sliver or a bumper, and from one the model accepted a bush as a car. It writes verify_accepted, verify_label, verify_confidence and verify_visible_fraction; keep_rejected=True keeps the rejects flagged and reject_unverified=True drops what it could not judge.
9. A model settles the pairs geometry cannot. One vehicle reported twice, a long vehicle seen as cab and box, a track split by a corner, two passes geometry could not join, sits a few metres from its twin with good crops each, and no geometric rule separates it from two cars parked side by side. DuplicateResolver (stage VERIFY, after the verifier) shows the model the crops of every pair of objects within max_distance_m (8 m), zoomed to their outlines, makes it name each vehicle's type and colour, and merges the pairs it calls one vehicle at min_confidence (0.9) or more. The object seen in the most frames is kept with the others' sightings and crops (duplicate_merged). On the labelled block the full chain went from 81 % precision and 97 % recall to 95 % and 92 %, with no duplicates left.

Parameter	Default	Notes
classes	required	'vehicles', 'utility poles', or 'object--vehicle--truck'.
detection_source	'auto'	Mapillary detections, map features for static point classes, SAM 3 on thumbnails for classes Mapillary lacks (debris piles); 'mapillary' or 'sam3' force one.
start_date, end_date	''	Inclusive capture-date window; use one survey at a time, since a car in the same driveway on another day may be a different car.
frame_spacing_m	0.0	Keep every frame; 3.0 is plenty for triangulation. Raise it generously in SAM 3 mode, where each frame costs an inference.
min_observations	1	3 removes most noise on a survey sampled every few metres.
camera_height_m	2.4	The RAPID rig; shapes the single-view range prior only.
localization_method	'tracks'	'voting' (ray voting on a grid) and 'cluster' (the earlier radius clustering) are kept for comparison.
merge_pieces_of_neighbours	True	Step 5.
close_range_m, min_sighting_deg, report_single_views	20.0, 6.0, False	Step 6: far sightings of a placed vehicle, specks and single views are not reported. 0 on the first two keeps everything.
abeam_window_deg	0.0 (off)	Build objects from the passing frames only, within this many degrees of abeam, and attach the frames looking down the road afterwards. Off until the track classification is reworked for passes alone; a 75-degree window lost real vehicles.
reid, reid_model, reid_min_similarity	False, CLIP ViT-B/16, per backbone (0.89 CLIP, 0.83 DINOv2)	Step 7.
reid_max_distance_m, reid_min_width_px, reid_image_size	6.0, 100, '2048'	Which objects appearance may compare, and the thumbnail its crops come from.
max_range_m, max_single_view_range_m	60.0, 30.0	Farthest reported position; farthest single view kept.
witness_radius_m	12.0	Weak estimates that nearby same-day frames should have seen but did not are dropped.
ego_filter, ego_min_recurrence	True, 0.5	Survey-vehicle removal.
frame_batch_size, simplify_tolerance	200, 0.002	Bound memory on city-wide runs; outlines are simplified on arrival.

Each object is a point PhysicalAsset with asset_type, localization, n_observations, n_images, position_sigma_m, parallax_deg and the sightings as Observation dictionaries under attributes['observations']. MapillaryObjectImageExtractor reads those records, keeps the two widest views per object (view_selection='widest'; 'closest' ranks by range instead), downloads each source image once at image_size, crops every object seen in it and releases it. Crop properties include image_id, captured_at, label, confidence, range_m, bearing, crop_box, source_image_size and outline (the detection polygon in crop pixels).

Sizing a run: the 1 km Spokane neighbourhood above, restricted to the August 2026 survey (5,250 frames after thinning), takes about 6 minutes and 1.3 GB of RAM for roughly 650 vehicles before verification and about 450 after; most of the time is reading detections from the Graph API. The whole survey (116,000 frames after thinning) needs about 75 minutes just to read detections.

Common pitfalls

• ValueError: Provide a Mapillary access_token. Pass access_token= (or a MapillaryClient); api_key= is an accepted synonym.
• No Street View image for an asset. Raise search_radius_m; coverage is sparse off main roads, and the endpoints are unofficial.
• Mapillary returns nothing. Check the date strings (YYYY-MM-DD), and try filter_rapid_only=False outside RAPID survey areas.
• strict_content_filter=True skips everything. The asset type must be in Mapillary's vocabulary; supply asset_type_mapping or a label_mapper.
• The survey car shows up as hundreds of vehicles. Keep ego_filter=True; the filter needs sequences of several frames, so do not thin to a handful.
• Too many single_view objects. Lower frame_spacing_m or raise min_observations to 3; filter with filter_by_attribute('localization', 'triangulated').
• The same vehicle is reported twice a few metres apart. Within one pass this is a split outline, a far sighting or a track that slid onto a neighbour; the default merge_pieces_of_neighbours and close_range_m handle most of it, and DuplicateResolver after the verifier settles what geometry cannot (a long vehicle seen from two sides, a track split by a corner). Across passes with a GPS offset larger than the merge gate, turn on reid=True. Across survey days, restrict start_date and end_date to one survey instead: the package never assumes a car stayed put.
• Obvious vehicles in driveways are missing. They were probably never seen within 20 m and sat within 15 m of a kerb-parked car; lower close_range_m or set it to 0, and check whether the verifier rejected them (keep_rejected=True, verify_label).
• DetectionVerifier removed real vehicles, or let junk through. It rejects anything less than half visible by default; lower min_visible_fraction or pass keep_rejected=True and inspect verify_label. Junk getting through usually means it saw the closest crops rather than the widest: keep the cropper's view_selection='widest'.
• DuplicateResolver merged two different vehicles. It only acts at min_confidence 0.9 or more and names each vehicle's type and colour first; raise the threshold or pass keep_duplicates=True to see the pairs it joined (duplicate_of).
• Memory grows on large regions. Narrow the region or dates first; keep frame_batch_size and simplify_tolerance at their defaults.
• Analyzer sees both surveys. Use image_filter on the save path or image ID to pick one date window.
• Bing Streetside. Not available without a key; only Bing aerial tiles are keyless.

See also Aerial Imagery Workflows, Vision-Language Models, Data Formats and Export and Examples.