Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added
- Common: `CameraParams` (intrinsic `K`, per-frame `world_to_camera`, width, height) and the optional `SensorFrame.camera_params` field — camera geometry a fusion strategy needs to project world-frame `lidar` into the image. Extrinsic targets the OpenCV optical frame (x right, y down, z forward). Additive: `WorldBridge.get_camera_params()` defaults to `None`, so no existing bridge changes; bridges declaring `CAMERA_RGB` / `CAMERA_DEPTH` should override it
- Execution: `c40_start_pose` (`[x, y, theta]` or null) — profile-defined ego start; factory falls back to the global-plan start point when null
- Perception: `State.set_start()` — capture current pose as the snapshot restored by `reset()` (via `get_copy` / `copy_from`)
- Visualizer: **Save Start** on the Execution state row — writes live ego pose into `c40_start_pose` and the active profile YAML
- Plugin registry: optional `display_name` — human-readable plugin title in the Plugins browser and the docs store; falls back to `name`, which stays the install-folder / import identifier
- Plugin registry: optional `site_url` — **Open Website** button plus a Website row in the plugin details window, and a **Site** button on the docs plugin cards

### Changed
- Perception: `State` / `AgentState` reset snapshot is a polymorphic copy of all fields (drops per-field `__init_*` / `AgentState.reset` override)
- Execution: `BasicSim.reset()` restores ego and NPC start poses (and NPC controllers) instead of clearing agents
- Execution: drop duplicate `world.reset()` call in `ExecutionStrategy.reset()`
- Docs / README: Tk visualizer demo uses a looping video (`docs/imgs/tk_visualizer.mp4`) instead of a static screenshot; landing shot fills the content column
- Docs: call out pause / step / interactive debug early (landing value strip, Overview features, Quick Start)
- Docs: Community Plugins cards are no longer whole-card GitHub links — explicit **Site** / **Repo** buttons sit above the GitHub stats footer, dependency notes clamp to two lines (full text on hover), and the links carry per-plugin aria-labels
- Docs: plugin registry field tables list every field with a required column (README, Overview, Plugin Development)

### Fixed
- Execution: `AsyncThreadedExecuter.step` no longer recreates workers while `stopped` is set — `StopExecAtGoalTask` (and any cooperative stop) stays stopped instead of being cleared by the next poll
- Visualizer: Execution loop mirrors `executer.stopped` into `exec_running` / Stop UI so task-driven stops end the Tk run (headless already checked the flag)
- Execution: async combined perception/planning worker localizes and perceives before replan on the shared snapshot — same order as `SyncExecuter`, so planning sees this iteration's ego/obstacles
- Common: `TrajectoryTracker` / `slice_trajectory_horizon` tolerate a 1-point path (final waypoint) — Frenet conversion no longer indexes `next_wp=1` and crashes `VelocityLocalPlanner.replan` at path end
- Visualizer: Control **Step** / Steer / Accel apply plant control and sync stack PM via `apply_world_control` (same dual-write as teleport after the world/stack ego split)
- Visualizer: **Save Start** snapshots velocity 0 so Reset matches a cold profile start (live speed is preserved while driving)
- Execution: perception, planning, and control share one `SensorFrame` per tick instead of each fetching its own — the stack no longer assumes the world holds still between stages, so bridges whose sensors evolve independently (CARLA async mode) stay coherent. `_localization_step` / `_perception_step` / `_replan_step` / `_control_step` now take the snapshot as an argument; each executer loop resolves its pacing gates first and fetches at most once (skipping the fetch entirely when no stage is due)
- Visualizer: Control **Align** teleports plant ego and syncs stack PM (stack-only writes were undone by GT localization after the world/stack ego split)
- Common: `TrajectoryTracker.update_waypoint_by_wp` / `update_to_next_waypoint` clamp `next_wp` at the path end — `%` precedence previously left `next_wp == len(path)` and crashed plot/step at the final waypoint
- Common: `create_quintic_trajectory_sd` b-vector matches the constraint matrix (end 1st / start 2nd derivatives were swapped)

## [0.5.3] - 2026-07-24

### Added
Expand Down
13 changes: 9 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
<a href="https://pypi.org/project/avlite/"><img alt="PyPI" src="https://img.shields.io/pypi/v/avlite?color=blue&logo=pypi&logoColor=white"></a>
<a href="https://pypi.org/project/avlite/"><img alt="Downloads" src="https://img.shields.io/pypi/dm/avlite?color=blue&logo=pypi&logoColor=white"></a>
<a href="https://github.com/AV-Lab/avlite/blob/main/pyproject.toml"><img alt="Python" src="https://img.shields.io/badge/python-3.10%2B-blue?logo=python&logoColor=white"></a>
<a href="https://avlab.io/avlite/"><img alt="Documentation" src="https://img.shields.io/badge/docs-avlab.io-orange?logo=materialformkdocs&logoColor=white"></a>
<a href="https://avlite.org/"><img alt="Documentation" src="https://img.shields.io/badge/docs-avlite.org-orange?logo=materialformkdocs&logoColor=white"></a>
<a href="https://github.com/AV-Lab/avlite/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/github/license/AV-Lab/avlite?color=green"></a>
</p>

Expand All @@ -39,7 +39,7 @@
<a href="#installation">Install</a> ·
<a href="#quick-start">Quick Start</a> ·
<a href="#architecture-overview">Architecture</a> ·
<a href="https://avlab.io/avlite/">Documentation</a> ·
<a href="https://avlite.org/">Documentation</a> ·
<a href="docs/plugin-development.md">Plugins</a> ·
<a href="#community-plugins">Community</a>
</p>
Expand All @@ -51,7 +51,10 @@
> **ROS2 & Autoware Ready** — Optional ROS2 executer plugin with native Autoware message support (`Trajectory`, `ControlCommand`, etc.).

<p align="center">
<img src="https://raw.githubusercontent.com/AV-Lab/avlite/main/docs/imgs/tk_visualizer.png" alt="AVLite Tk Visualizer" width="100%">
<video src="https://raw.githubusercontent.com/AV-Lab/avlite/main/docs/imgs/tk_visualizer.mp4"
poster="https://raw.githubusercontent.com/AV-Lab/avlite/main/docs/imgs/tk_visualizer.png"
controls muted autoplay loop playsinline width="100%">
</video>
</p>

## Architecture Overview
Expand Down Expand Up @@ -256,12 +259,14 @@ See the [Plugin Development Guide — Publish via pull request](docs/plugin-deve
```yaml
plugins:
- name: my_perception_plugin
display_name: My Perception Plugin # optional, defaults to name
description: One-line summary of what the plugin does
repository: https://github.com/your-org/your-plugin-repo
version: latest # or a tag/commit SHA
author: your-org
category:
- PerceptionStrategy
site_url: "" # optional project website
```

4. Open a pull request. Once merged it shows up in every user's
Expand Down Expand Up @@ -367,7 +372,7 @@ Distributed under the MIT License. See [`LICENSE`](LICENSE) for details.

<p align="center">
Built with care by <a href="https://github.com/AV-Lab">AV-Lab</a> ·
<a href="https://avlab.io/avlite/">Documentation</a> ·
<a href="https://avlite.org/">Documentation</a> ·
<a href="https://github.com/AV-Lab/avlite/issues">Report a bug</a> ·
<a href="https://github.com/AV-Lab/avlite/issues">Request a feature</a>
</p>
Expand Down
21 changes: 5 additions & 16 deletions avlite/c10_perception/c11_perception_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,11 +134,11 @@ class State:
length: float = 4.5

def __post_init__(self):
# initial x,y position, useful for reset
self.__init_x = self.x
self.__init_y = self.y
self.__init_theta = self.theta
self.__start = self.get_copy()

def set_start(self):
"""Capture the current state as the snapshot restored by :meth:`reset`."""
self.__start.copy_from(self)

def get_bb_corners(self) -> np.ndarray:
"""Get the bounding box corners of the vehicle in world coordinates."""
Expand Down Expand Up @@ -173,9 +173,7 @@ def get_bb_corners(self) -> np.ndarray:
return np.c_[rotated_corners_x, rotated_corners_y]

def reset(self):
self.x = self.__init_x
self.y = self.__init_y
self.theta = self.__init_theta
self.copy_from(self.__start)

def copy_from(self, other: State) -> None:
"""Copy dataclass fields from *other* in place (preserves object identity)."""
Expand Down Expand Up @@ -205,15 +203,6 @@ class AgentState(State):
agent_id: int = -1
agent_type: AgentType = AgentType.ACKERMANN

def __post_init__(self):
super().__post_init__()
self.__init_speed = self.velocity

def reset(self):
super().reset()
self.velocity = self.__init_speed



@dataclass
class EgoState(AgentState):
Expand Down
29 changes: 27 additions & 2 deletions avlite/c40_execution/c41_world_bridge.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
from avlite.c50_common.c53_stack_datatypes import control_type_for_agent
from avlite.c50_common.c52_world_sensor_datatypes import (
WORLD_CAPABILITY_SENSOR_FIELDS,
CameraParams,
GnssReading,
ImuReading,
SensorFrame,
Expand All @@ -20,6 +21,10 @@
RgbImage,
)

import logging

log = logging.getLogger(__name__)


@dataclass
class WorldBridge(ABC):
Expand Down Expand Up @@ -118,6 +123,15 @@ def get_depth_image(self, agent_id: int = EGO_AGENT_ID) -> DepthImage | None:
self._require_ego_agent(agent_id, "depth")
return None

def get_camera_params(self, agent_id: int = EGO_AGENT_ID) -> CameraParams | None:
"""Returns geometry for the rgb/depth camera. Layout: ``CameraParams``.

Bridges exposing CAMERA_RGB or CAMERA_DEPTH must override this; without
it, world-frame LiDAR cannot be projected into the image.
"""
self._require_ego_agent(agent_id, "camera params")
return None

def get_lidar_data(self, agent_id: int = EGO_AGENT_ID) -> LidarCloud | None:
"""Returns the lidar point cloud. Layout: ``LidarCloud`` in c52_world_sensor_datatypes."""
self._require_ego_agent(agent_id, "lidar")
Expand Down Expand Up @@ -145,6 +159,7 @@ def get_sensor_frame(self, agent_id: int = EGO_AGENT_ID) -> SensorFrame:
frame = SensorFrame(
rgb=self.get_rgb_image(),
depth=self.get_depth_image(),
camera_params=self.get_camera_params(),
lidar=self.get_lidar_data(),
imu=self.get_imu(),
gnss=self.get_gnss(),
Expand All @@ -154,13 +169,19 @@ def get_sensor_frame(self, agent_id: int = EGO_AGENT_ID) -> SensorFrame:
frame = SensorFrame(
rgb=self.get_rgb_image(agent_id=agent_id),
depth=self.get_depth_image(agent_id=agent_id),
camera_params=self.get_camera_params(agent_id=agent_id),
lidar=self.get_lidar_data(agent_id=agent_id),
imu=self.get_imu(agent_id=agent_id),
gnss=self.get_gnss(agent_id=agent_id),
wheel_odometry=self.get_wheel_odometry(agent_id=agent_id),
)

log.debug("Sensor frame before world capability filter: %s", frame)
return self._apply_world_capability_filter(frame)

def reset(self):
pass

@staticmethod
def _apply_world_capability_filter(frame: SensorFrame) -> SensorFrame:
"""Null sensor fields whose world capabilities are disabled in Bridge Setting."""
Expand All @@ -177,6 +198,12 @@ def _apply_world_capability_filter(frame: SensorFrame) -> SensorFrame:
continue
setattr(frame, field, None)
cleared.add(field)
# Camera geometry describes rgb/depth: drop it when neither is provided.
if not any(
is_world_capability_enabled(c)
for c in (WorldCapability.CAMERA_RGB, WorldCapability.CAMERA_DEPTH)
):
frame.camera_params = None
return frame

def _require_ego_agent(self, agent_id: int, method: str) -> None:
Expand All @@ -185,8 +212,6 @@ def _require_ego_agent(self, agent_id: int, method: str) -> None:
f"{type(self).__name__} does not support {method} for agent {agent_id}"
)

def reset(self):
pass

def __init_subclass__(cls, abstract=False, **kwargs):
super().__init_subclass__(**kwargs)
Expand Down
25 changes: 12 additions & 13 deletions avlite/c40_execution/c42_execution_strategy.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
TaskStrategy,
)
from avlite.c50_common.c51_capabilities import StackCapability, satisfies_requirements
from avlite.c50_common.c52_world_sensor_datatypes import SensorFrame
from avlite.c50_common.c56_fps_tracker import FpsTracker

log = logging.getLogger(__name__)
Expand Down Expand Up @@ -156,7 +157,6 @@ def reset(self):
self.local_planner.reset()
if self.controller:
self.controller.reset()
self.world.reset()
self.elapsed_real_time = 0
self.elapsed_sim_time = 0
self._last_cmd = None
Expand Down Expand Up @@ -252,15 +252,14 @@ def _can_actuate(self) -> bool:

# --- tick helpers ---

def _localization_step(self) -> None:
"""Run one localization iteration using the current world capabilities."""
def _localization_step(self, sensors: SensorFrame) -> None:
"""Run one localization iteration on the caller-supplied tick snapshot."""
if not self.localization:
return

world_ok = satisfies_requirements(self.localization.world_requirements, self.world.world_capabilities)
stack_ok = satisfies_requirements(self.localization.stack_requirements, self.available_stack_capabilities())
if world_ok and stack_ok:
sensors = self.world.get_sensor_frame()
self.localization.localize(perception_model=self.pm, sensors=sensors)
self.localization_fps = self._localization_fps_tracker.tick()
# Harvest optional stack_event stamp on PerceptionModel (notify once, then clear).
Expand All @@ -276,8 +275,8 @@ def _localization_step(self) -> None:
f"Skipping."
)

def _perception_step(self) -> None:
"""Run one perception iteration and update fps tracking."""
def _perception_step(self, sensors: SensorFrame) -> None:
"""Run one perception iteration on the caller-supplied tick snapshot."""
if not self.perception:
log.debug("Perception strategy is not set. Skipping perception step.")
return
Expand All @@ -304,7 +303,6 @@ def _perception_step(self) -> None:
else:
self.pm.agent_vehicles = []

sensors = self.world.get_sensor_frame()
self.perception.perceive(perception_model=self.pm, sensors=sensors)

self.perception_fps = self._perception_fps_tracker.tick()
Expand All @@ -314,11 +312,10 @@ def _perception_step(self) -> None:
self.pm.stack_event = None
self.task_runner.notify(event)

def _replan_step(self) -> None:
"""Run one planning iteration (replan) and update FPS."""
def _replan_step(self, sensors: SensorFrame) -> None:
"""Run one planning iteration (replan) on the caller-supplied tick snapshot."""
if not self.local_planner:
return
sensors = self.world.get_sensor_frame()
self.local_planner.replan(perception_model=self.pm, sensors=sensors)
self.planner_fps = self._planner_fps_tracker.tick()
# Harvest optional stack_event stamps from plan artifacts (notify once, then clear).
Expand All @@ -338,13 +335,15 @@ def _replan_step(self) -> None:
gp.stack_event = None
self.task_runner.notify(event)

def _control_step(self, sim_dt: float) -> None:
"""Recompute control command into ``_last_cmd`` (no world integrate)."""
def _control_step(self, sim_dt: float, sensors: SensorFrame) -> None:
"""Recompute control command into ``_last_cmd`` (no world integrate).

Uses the caller-supplied tick snapshot.
"""
if not self.controller or not self.local_planner:
return
if not self._can_actuate():
return
sensors = self.world.get_sensor_frame()
local_plan = self.local_planner.get_local_plan()
cmd = self.controller.control(
self.pm.ego_vehicle, local_plan, control_dt=sim_dt,
Expand Down
Loading