Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 26 additions & 15 deletions docs/simulation/isaac_sim/overhead_camera.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,25 +70,38 @@ The three constants (`OVERHEAD_ALTITUDE_M`, `OVERHEAD_COVERAGE_M`, `OVERHEAD_PX_
| `OVERHEAD_COVERAGE_M` | `200.0` | Side length of the captured square (m). |
| `OVERHEAD_PX_PER_METER` | `4.0` | Texture density. Increase for sharper text/markings; capped at `max_resolution=2048`. |

The camera is positioned at world origin `(0, 0)`. If your scene's points of interest are off-origin, shift the camera's `prim_path` xform after `add_orthographic_camera` returns:
### Re-centering or transforming the camera

By default the camera sits over world origin `(0, 0)`. For an off-origin area of interest, pass `center_x_m` / `center_y_m` to both helpers — they take care of the camera xform and the spec topics the GCS reads:

```python
from pxr import Gf, UsdGeom
CENTER_X_M, CENTER_Y_M = 50.0, -25.0

cam_path = add_orthographic_camera(stage, prim_path="/World/MapCamera", ...)
cam_path = add_orthographic_camera(
stage, prim_path="/World/MapCamera",
altitude_m=OVERHEAD_ALTITUDE_M,
coverage_m=OVERHEAD_COVERAGE_M,
scene_scale_factor=scene_scale_factor,
center_x_m=CENTER_X_M,
center_y_m=CENTER_Y_M,
)

# Re-center the camera over (CENTER_X_M, CENTER_Y_M) instead of world origin.
CENTER_X_M, CENTER_Y_M = 50.0, -25.0
xform = UsdGeom.Xformable(stage.GetPrimAtPath(cam_path))
xform.ClearXformOpOrder()
xform.AddTranslateOp().Set(Gf.Vec3d(
CENTER_X_M * scene_scale_factor,
CENTER_Y_M * scene_scale_factor,
OVERHEAD_ALTITUDE_M * scene_scale_factor,
))
add_overhead_camera_publisher(
parent_graph_path="/World/MapCameraGraph",
camera_prim_path=cam_path,
topic="/sim/overhead/image",
spec_topic="/sim/overhead/spec",
center_x_topic="/sim/overhead/center_x",
center_y_topic="/sim/overhead/center_y",
frame_id="map",
coverage_m=OVERHEAD_COVERAGE_M,
center_x_m=CENTER_X_M,
center_y_m=CENTER_Y_M,
pixels_per_meter=OVERHEAD_PX_PER_METER,
domain_id=0,
)
```


## GCS side

The GCS rendering is handled by `_build_sim_ground_marker` in `gcs/ros_ws/src/gcs_visualizer/gcs_visualizer/foxglove_visualizer_node.py`. It:
Expand All @@ -113,8 +126,6 @@ The default downsample (0.8 cells/m, cap 384) is conservative. To raise the rend
<param name="overhead_max_grid_resolution" value="1024" />
```

`1024` over a 200 m scene gives ~5 cells/m. If the 3D panel slows down, drop back toward `768`. Any change here also requires bumping `OVERHEAD_PX_PER_METER` on the sim side (otherwise you're sampling a low-resolution source more densely).

To change other rendering behavior (alpha, lighting), edit `_build_sim_ground_marker` directly. To force a re-render, restart the GCS visualizer.

## See also
Expand Down
92 changes: 92 additions & 0 deletions docs/simulation/isaac_sim/spawning_drones.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,98 @@ set_gps_origins(DRONE_CONFIGS, world_origin=(40.4433, -79.9436, 280.0)) # Pitts

The anchor only affects the geographic location reported via GPS; nothing in the scene moves. Pick something close to where you want the drones to "be" — Foxglove's Map panel will center on it, and any GPS-referenced inputs to your stack will be relative to it.

## Scene prep helpers — `scene_prep.py`

`simulation/isaac-sim/utils/scene_prep.py` is the small toolbox of stage preparation helpers `example_multi_drone_scene_import.py` uses inside its post-load callback (after the stage is loaded, before drones spawn). The full file has more — what's documented here is what you'll reach for in 95% of scenes.

```python
from utils.scene_prep import (
get_stage_meters_per_unit, scale_stage_prim, add_colliders,
add_dome_light, save_scene_as_contained_usd,
add_orthographic_camera, add_overhead_camera_publisher,
)

mpu, scene_scale_factor = get_stage_meters_per_unit(stage)
```

### Scaling — `scale_stage_prim`

USD scenes are authored at all sorts of stage units. To apply a uniform scale to the imported stage root once, before drones spawn:

```python
STAGE_SCALE = 0.01 # cm → m
scale_stage_prim(stage, "/World/stage", STAGE_SCALE)
```

### Colliders — `add_colliders`

Recursively applies `UsdPhysics.CollisionAPI` to every `UsdGeom.Mesh` under the given prim. Imported environment USDs are usually visual-only; without this, drones fall through buildings.

```python
stage_prim = stage.GetPrimAtPath("/World/stage")
add_colliders(stage_prim)
```

Skips prims that already have the API applied. Run it on the stage root after `scale_stage_prim` returns.

### Lighting — `add_dome_light`

Incase the scene is missing any lights, this adds a dome light that can act like an overhead 'sun'.

```python
add_dome_light(
stage,
prim_path="/World/DomeLight",
intensity=3500.0,
exposure=-5.0, # negative = darker; tune per scene
)
```

### Overhead camera — `add_orthographic_camera` + `add_overhead_camera_publisher`

Used as a pair: one drops an orthographic camera over the scene, the other wires an OmniGraph to publish its frame plus three Float32 spec topics (`coverage_m`, `center_x_m`, `center_y_m`) the GCS uses to texture a ground plane in Foxglove's 3D panel.

```python
cam_path = add_orthographic_camera(
stage,
prim_path="/World/MapCamera",
altitude_m=165.0,
coverage_m=225.0,
scene_scale_factor=scene_scale_factor,
center_x_m=0.0, # set if your area of interest isn't at world origin
center_y_m=0.0,
)
add_overhead_camera_publisher(
parent_graph_path="/World/MapCameraGraph",
camera_prim_path=cam_path,
topic="/sim/overhead/image",
spec_topic="/sim/overhead/spec",
center_x_topic="/sim/overhead/center_x",
center_y_topic="/sim/overhead/center_y",
frame_id="map",
coverage_m=225.0,
center_x_m=0.0,
center_y_m=0.0,
pixels_per_meter=10.0,
domain_id=0,
)
```

Full setup, GCS-side rendering, and tuning knobs are on the **[Overhead Camera](overhead_camera.md)** page.

### Saving a self-contained copy — `save_scene_as_contained_usd`

For scenes you'd like to keep working with offline (no Nucleus connection), or for sharing a scene with collaborators, collect the root USD plus every referenced asset (textures, MDLs, sublayers) into a local directory:

```python
save_scene_as_contained_usd(
source_usd_url=ENV_URL,
output_dir="/tmp/collected_scene",
)
```

The collected folder contains a standalone root USD with relative references — load it directly via `omniverse://localhost/...` or a local file path. The collected scene will include modifications for scale, colliders, etc applied before saving.

## Common issues

| Symptom | Likely cause | Fix |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
from pegasus.simulator.logic.interface.pegasus_interface import PegasusInterface
from pegasus.simulator.ogn.api.spawn_multirotor import spawn_px4_multirotor_node
from pegasus.simulator.ogn.api.spawn_zed_camera import add_zed_stereo_camera_subgraph
from pegasus.simulator.ogn.api.spawn_ouster_lidar import add_ouster_lidar_subgraph
from pegasus.simulator.ogn.api.spawn_rtx_lidar import add_rtx_lidar_subgraph

# gps_utils lives in the same directory as this script
_LAUNCH_SCRIPTS_DIR = os.path.dirname(os.path.abspath(__file__))
Expand All @@ -39,7 +39,7 @@
import scene_prep
from scene_prep import (
scale_stage_prim, add_colliders, add_dome_light, get_stage_meters_per_unit,
reference_root_prims_under_world,
reference_root_prims_under_world, dedupe_physics_scenes,
add_orthographic_camera, add_overhead_camera_publisher,
)

Expand All @@ -49,9 +49,7 @@

#env/stage path and scale
ENV_URL = f"omniverse://{NUCLEUS_SERVER}/Projects/AirStack/scenes/urban/allegheny_county_fire_academy/fire_academy.scene.usd"
#f"omniverse://{NUCLEUS_SERVER}/Library/Assets/FireAcademyFaro/fire_academy_faro.usd"
#f"omniverse://{NUCLEUS_SERVER}/Projects/AirStack/RayFronts-Planner/FireAcademy.scene.usd"
#f"omniverse://{NUCLEUS_SERVER}/Library/Assets/Fire_Academy_Digital_Twin/fire_academy.usd"

STAGE_SCALE = 0.01

DRONE_USD = "~/.local/share/ov/data/documents/Kit/shared/exts/pegasus.simulator/pegasus/simulator/assets/Robots/Iris/iris.usd"
Expand All @@ -74,21 +72,27 @@
# spawn location for /Assets/Fire_Academy_Digital_Twin/fire_academy.usd:
# {"domain_id": 1, "x_m": 20.0, "y_m": -7.0, ...}
# {"domain_id": 2, "x_m": 17.0, "y_m": 1.5, ...}

SPAWN_HEIGHT_ABOVE_FLOOR_M = 0.03
DRONE_CONFIGS = [
{"domain_id": 1, "x_m": 27.0, "y_m": 7.6, "z_m": SPAWN_HEIGHT_ABOVE_FLOOR_M, "orient": [0.0, 0.0, -0.937, 0.35], "lidar_min_range": 4.0},
{"domain_id": 2, "x_m": 23.0, "y_m": 9.8, "z_m": SPAWN_HEIGHT_ABOVE_FLOOR_M, "orient": [0.0, 0.0, -0.937, 0.35], "lidar_min_range": 4.0},
{"domain_id": 3, "x_m": 27.0, "y_m": 12.0, "z_m": SPAWN_HEIGHT_ABOVE_FLOOR_M, "orient": [0.0, 0.0, -0.937, 0.35], "lidar_min_range": 4.0},
{"domain_id": 4, "x_m": 23.0, "y_m": 14.0, "z_m": SPAWN_HEIGHT_ABOVE_FLOOR_M, "orient": [0.0, 0.0, -0.937, 0.35], "lidar_min_range": 4.0}
]

# Top-down "map" camera over (0, 0). Captures one aerial of the static scene
# that the GCS visualizer turns into a textured ground in Foxglove's 3D panel.
OVERHEAD_ALTITUDE_M = 150.0
OVERHEAD_COVERAGE_M = 200.0 # per-map knob: world meters per side.
OVERHEAD_PX_PER_METER = 12.0 # Source-image density. Bump for sharper texture.
{"domain_id": 1, "x_m": 32.0, "y_m": 12.6, "z_m": SPAWN_HEIGHT_ABOVE_FLOOR_M, "orient": [0.0, 0.0, -0.937, 0.35], "lidar_min_range": 4.0},
{"domain_id": 2, "x_m": 28.0, "y_m": 14.8, "z_m": SPAWN_HEIGHT_ABOVE_FLOOR_M, "orient": [0.0, 0.0, -0.937, 0.35], "lidar_min_range": 4.0},
{"domain_id": 3, "x_m": 32.0, "y_m": 19.8, "z_m": SPAWN_HEIGHT_ABOVE_FLOOR_M, "orient": [0.0, 0.0, -0.937, 0.35], "lidar_min_range": 4.0}
]

# Top-down "map" camera. Captures one aerial of the static scene that the
# GCS visualizer turns into a textured ground in Foxglove's 3D panel. The
# camera centers on (OVERHEAD_CENTER_X_M, OVERHEAD_CENTER_Y_M) in world
# meters — leave both 0.0 for the legacy origin-centered behavior.
OVERHEAD_ALTITUDE_M = 165.0
OVERHEAD_COVERAGE_M = 225 # per-map knob: world meters per side.
OVERHEAD_CENTER_X_M = 0.0 #-152 # world-X of camera center / texture center.
OVERHEAD_CENTER_Y_M = 0.0 #-80 # world-Y of camera center / texture center.
OVERHEAD_PX_PER_METER = 10.0 # Source-image density. Bump for sharper texture.
OVERHEAD_TOPIC = "/sim/overhead/image"
OVERHEAD_SPEC_TOPIC = "/sim/overhead/spec"
OVERHEAD_CENTER_X_TOPIC = "/sim/overhead/center_x"
OVERHEAD_CENTER_Y_TOPIC = "/sim/overhead/center_y"
OVERHEAD_FRAME_ID = "map"
OVERHEAD_DOMAIN_ID = 0
# ---------------------------------------------------------
Expand Down Expand Up @@ -165,8 +169,13 @@ def __init__(self):
if not wait_for_stage(stage):
carb.log_warn("Stage load timed out — continuing anyway.")

dedupe_physics_scenes(stage)

# ----- Scene preparation -----
# Bring in sky/sun/environment prims that sit outside /World in the source USD
# Bring in sky/sun/environment prims that sit at root level in the
# source USD next to the defaultPrim that pg.load_environment already
# loaded into /World/stage. reference_root_prims_under_world skips
# the defaultPrim, so this can't duplicate geometry.
reference_root_prims_under_world(stage, ENV_URL)

stage_prim = stage.GetPrimAtPath("/World/stage")
Expand All @@ -193,14 +202,20 @@ def __init__(self):
altitude_m=OVERHEAD_ALTITUDE_M,
coverage_m=OVERHEAD_COVERAGE_M,
scene_scale_factor=s,
center_x_m=OVERHEAD_CENTER_X_M,
center_y_m=OVERHEAD_CENTER_Y_M,
)
add_overhead_camera_publisher(
parent_graph_path="/World/MapCameraGraph",
camera_prim_path=cam_path,
topic=OVERHEAD_TOPIC,
spec_topic=OVERHEAD_SPEC_TOPIC,
center_x_topic=OVERHEAD_CENTER_X_TOPIC,
center_y_topic=OVERHEAD_CENTER_Y_TOPIC,
frame_id=OVERHEAD_FRAME_ID,
coverage_m=OVERHEAD_COVERAGE_M,
center_x_m=OVERHEAD_CENTER_X_M,
center_y_m=OVERHEAD_CENTER_Y_M,
pixels_per_meter=OVERHEAD_PX_PER_METER,
domain_id=OVERHEAD_DOMAIN_ID,
)
Expand Down Expand Up @@ -230,14 +245,15 @@ def __init__(self):
camera_rotation_offset=[0.0, 0.0, 0.0],
)

add_ouster_lidar_subgraph(
add_rtx_lidar_subgraph(
parent_graph_handle=graph_handle,
drone_prim=f"/World/drone{i}/base_link",
robot_name=f"robot_{i}",
lidar_name="OS1_REV6_128_10hz___512_resolution",
lidar_config="ouster_os1",
lidar_topic_name="point_cloud_raw",
lidar_offset=[0.0, 0.0, 0.025],
lidar_rotation_offset=[0.0, 0.0, 0.0],
lidar_min_range=cfg["lidar_min_range"],
min_range=cfg["lidar_min_range"],
)

self.play_on_start = os.environ.get("PLAY_SIM_ON_START", "true").lower() == "true"
Expand Down
Loading
Loading