feat: add 7 new canonical builtin object types#90
Merged
Conversation
Wire-format contracts for the byte-backed builtin object types that PlotJuggler plugins emit and renderers consume: - Image, DepthImage, PointCloud: pair with existing SDK structs - CompressedPointCloud, OccupancyGrid: new byte-backed types - SceneEntities + 8 scene primitives (Arrow, Cube, Sphere, Cylinder, Line, Triangle, Text, Axes) for marker-style 3D visualization - Mesh3D, VideoFrame: opaque-asset family (binary payload + format identifier) Schema cleanup: - Add Pose to Geometry.proto (shared by scene primitives, OccupancyGrid, Mesh3D) - Merge FrameTransform.proto into FrameTransforms.proto to match the single-file-per-family pattern - Add pj/README.md describing the purpose of each schema Tooling: - Extend .clang-format with a Proto language section (Google base, 120-col limit) and enable the existing clang-format pre-commit hook on .proto files. - Pass --allow-multiple-documents to the check-yaml hook so it accepts the multi-doc .clang-format file. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Catalog additions matching the proto contracts landed in f37b7e4. This is the "top-level" change: BuiltinObjectType enum + SDK struct headers. Codecs follow in subsequent commits. New SDK structs: - sdk::OccupancyGrid: 2D metric grid (maps, costmaps) - sdk::CompressedPointCloud: opaque compressed cloud (Draco, ...) - sdk::Mesh3D: binary mesh asset (GLTF/STL/PLY/OBJ/USD/DAE) - sdk::VideoFrame: single frame of h264/h265/vp9/av1 stream - sdk::SceneEntities + 8 primitives (Arrow, Cube, Sphere, Cylinder, Line, Triangle, Text, Axes) for marker-style 3D visualization Also adds sdk::Pose to FrameTransforms.hpp (consumed by Mesh3D, OccupancyGrid, and the scene primitives). Long-term it belongs in a dedicated Geometry.hpp alongside Vector3/Quaternion/Point2/Point3 - separate refactor. BuiltinObject.hpp: - Enum gains 5 entries (kOccupancyGrid through kSceneEntities at numeric values 7..11; preserves the historical gap at 2). - name(), parseBuiltinObjectType(), and typeOf() extended to cover every new entry. Tests: builtin_object_test.cpp gains a round-trip test that iterates every enum value through name() / parseBuiltinObjectType() to catch future drift between the three helpers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hand-rolled protobuf codecs implementing the wire contracts landed in f37b7e4 for the SDK structs landed in 620f3da. Round-trip tests follow in a separate commit. New codecs (each with serialize / deserialize / kSchema constant): - video_frame_codec - compressed_point_cloud_codec - occupancy_grid_codec - mesh3d_codec - scene_entities_codec (also covers the 8 scene primitives: Arrow, Cube, Sphere, Cylinder, Line, Triangle, Text, Axes) Codec infrastructure: - protobuf_wire.hpp gains a parseFields(reader, handler) helper that drains the standard tag-read / switch / skip-unknown decode loop — cuts ~10 lines of boilerplate per nested message. - protobuf_wire.hpp gains fixed32 / packedFixed32 / readFixed32 / readPackedFixed32 for LinePrimitive.indices and similar. - Reader.readBytes is promoted from private to public so byte-backed codecs can copy bytes into their owned vector + BufferAnchor. - New internal geometry_codec.hpp consolidates the shared writeTimestamp / writeVector3 / writeQuaternion / writePose / writeColor / writePoint3 + decode counterparts. Pulled out of frame_transforms_codec.cpp; frame_transforms now includes it. Pure code move — wire format unchanged, existing tests pass. Existing image_annotations_codec.cpp is NOT refactored in this PR (it has its own writeColor / decodeColor that match byte-identically). That refactor is left for a follow-up to keep this PR scoped. Build wire-up: 5 new .cpp files added to pj_base library sources. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The SDK structs predate the catalog rework; their wire contracts landed in f37b7e4. This commit closes the codec gap so all byte-backed builtin types have round-trippable serialization. New codecs (each with serialize / deserialize / kSchema): - image_codec — kSchemaImage = "PJ.Image" - depth_image_codec — kSchemaDepthImage = "PJ.DepthImage" - point_cloud_codec — kSchemaPointCloud = "PJ.PointCloud" Codec infrastructure additions to protobuf_wire.hpp: - Writer.floatField + Reader.readFloat for `optional float` (used by Image.compressed_depth_min / _max). - Writer.packedDouble (range-based) for `repeated double` fields (DepthImage.K, DepthImage.D). - readPackedDouble / readPackedDoubleArray<N> free helpers for the decode side. The fixed-size-array variant guards DepthImage.K's 9-entry contract at the wire boundary. Misc: media_metadata_test.cpp uses "PJ.VideoFrame" in its example metadata payload instead of the now-renamed "PJ.CompressedVideo". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
AssetVideo carries no pixel data — it references a video file by path and surfaces decode-routing metadata (MIME type, dimensions, frame rate) so consumers can size playback windows and route to a decoder without opening the file. Producers (LeRobot dataset loaders, MP4 loaders) push exactly one AssetVideo per topic; the ObjectStore timestamp of that entry equals `time_origin_ns` so timeline UIs see the asset's start instant. Distinct from VideoFrame (which carries a single coded frame as opaque bytes) — AssetVideo is the file-reference family alongside Mesh3D, not the streamed-payload family. Adds: - pj_base/proto/pj/AssetVideo.proto - pj_base/include/pj_base/builtin/AssetVideo.hpp (SDK struct) - pj_base/include/pj_base/builtin/asset_video_codec.hpp - pj_base/src/builtin/asset_video_codec.cpp - kAssetVideo = 12 in BuiltinObjectType + name/parse/typeOf entries - Tests extended to cover round-trip through the catalog helpers SDK-side `time_origin_ns` and `duration_ns` are std::optional — absent means "not aligned to wall clock" / "probe the file" respectively. The codec emits these fields only when set, and on decode leaves them as nullopt when the field is absent on the wire. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The 12 SDK builtin headers under `pj_base/include/pj_base/builtin/` were the only CamelCase filenames in a codebase that is otherwise snake_case throughout. Rename them to match (e.g. `Image.hpp` → `image.hpp`, `SceneEntities.hpp` → `scene_entities.hpp`); update all `#include` directives, `@file` Doxygen tags, and the include snippet in `docs/builtin_type.md`. Proto files keep their CamelCase names — they mirror the contained message name (e.g. `Image.proto` → `message Image`), a separate convention scope. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Each new codec gets a small gtest file covering its schema-name constant, the empty-buffer error path, and a realistic round-trip through serialize/deserialize. SceneEntities additionally exercises one entity carrying every primitive kind plus per-vertex colors and indices on a LinePrimitive; Image/DepthImage/Mesh3D cover their two mutually-exclusive shapes (compressed-depth vs raw, plumb_bob vs rectified, embedded asset vs URL). Hand-rolled protobuf builders (varint/tag/double/fixed32/length- delimited/string/bytes) plus geometry encoders (Timestamp, Vector3, Quaternion, Pose) are extracted into a shared `pj_base/tests/protobuf_wire_test_helpers.hpp`. The existing frame_transforms_codec_test.cpp and image_annotations_codec_test.cpp adopt it; their golden-byte assertions remain unchanged so they keep guarding the on-wire format. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Document the 6 new builtin types (OccupancyGrid, CompressedPointCloud, Mesh3D, VideoFrame, AssetVideo, SceneEntities) added on this branch: update the public-headers include block, extend the serialization- families table (the byte-backed views family grew from 3 to 7; the owned-values family gained SceneEntities and AssetVideo), add the new kEnum rows to the classification table, and write a brief per-type section in the same shape as the existing Image/DepthImage/PointCloud sections. Also expand the Conversion Examples table with the new source-side inputs each type covers (Draco clouds, nav_msgs/OccupancyGrid, URDF mesh resources, marker arrays, raw codec frames, MP4 dataset files). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
facontidavide
force-pushed
the
feat/builtin-protos
branch
from
b45b717 to
f0b5e16
Compare
Stage B of the pj_scene3D PJ4-integration plan. Adds a canonical SDK type for URDF / SDF / MJCF robot descriptions so the upcoming 3D widget can consume them through the same ObjectStore path as other builtin objects (PointCloud, FrameTransforms, Image, etc.). - New PJ::sdk::RobotDescription owned struct: timestamp_ns, topic, format (open-ended hint string, mirrors Image::encoding), text (raw source markup). Header lives at pj_base/builtin/robot_description.hpp (snake_case to match the rest of this branch). - BuiltinObjectType::kRobotDescription = 13, appended after kAssetVideo. The original branch assigned slot 7, which collided with the kOccupancyGrid = 7 assignment landing in this PR; renumbered up to preserve stable IDs for all already-assigned values. - New cases in name(), parseBuiltinObjectType(), typeOf() — symmetric with kFrameTransforms. - builtin_object_test gains typeOf coverage, the round-trip loop entry, ParsesRobotDescriptionTypeName, and RobotDescriptionRoundtripPreservesFields. 57/57 tests pass. - docs/builtin_type.md: new section explaining raw-text + format-hint rationale (open format space, no canonical codec, no embedded mesh bytes — mesh resolution is consumer-side), enum table row, conversion example for std_msgs/String on /robot_description. The producer (ParserROS in stage D) validates the text matches the declared format before emission so generic std_msgs/String topics on unrelated channels don't surface as phantom robot descriptions. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
facontidavide
changed the title
3D consumers (pj_scene3D's Scene3DDockWidget, RViz/Foxglove-style viewers) need the source coordinate frame to TF-transform points to a fixed render frame. Without this, a cloud emitted in the sensor frame cannot be positioned in world space when multiple frames coexist. The field is purely additive — defaulted to empty string, so existing 2D-only consumers and scalar emission are unaffected. The proto schema gets `string frame_id = 10;` at the next free wire tag; the codec round- trips it; the C++ struct picks it up in stable position. Ports commit 1a38059 from feat/robot-description-builtin onto the snake_case + codec layout of feat/builtin-protos. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PJ_builtin_object_type_t (the C ABI enum that pure-C plugins compile against for classify_schema) was stuck at the original 5 typed values plus a single commented-out reservation for OccupancyGrid. The C++ enum has since grown to 13 entries with the snake_case + AssetVideo + RobotDescription work. Without this sync, a pure-C plugin trying to declare any of OccupancyGrid / CompressedPointCloud / Mesh3D / VideoFrame / SceneEntities / AssetVideo / RobotDescription via classify_schema has no way to name the value — it would silently map to PJ_BUILTIN_OBJECT_TYPE_NONE. Adds all 7 missing typed values at their stable numeric slots (matching BuiltinObjectType in builtin_object.hpp exactly) and pins each value with a static_assert in abi_layout_sentinels_test so a future renumber breaks the build. Also drops the obsolete 5-type list from the docstring in favor of pointing readers at the C++ enum as the source of truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Collaborator
|
For my understanding, and for tracking. What is the goal of the .proto files? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extends PlotJuggler's canonical builtin object catalog (the shim between
source-specific message families and PJ's internal data shapes) with six
new types geared toward ROS-style navigation, manipulation, and video
playback workloads.
New
BuiltinObjectTypeentries:kOccupancyGrid,kCompressedPointCloud,kMesh3D,kVideoFrame,kSceneEntities,kAssetVideo. Inclusion followsthe two filters discussed up front: not a scalar timeseries (JointState,
IMU, Wrench remain out) and the renderer treats them differently from any
existing type (LaserScan folds into PointCloud, Path folds into a
SceneEntities LinePrimitive).
What landed, by commit:
f37b7e4.protoschemas underpj_base/proto/pj/(wire-format contracts; no protobuf runtime dep)620f3daPoseadded toframe_transforms.hpp;BuiltinObject.hppenum +name()/parseBuiltinObjectType()/typeOf()extended;builtin_object_test.cppcovers all 12 enum entries374f170geometry_codec.hpp(Timestamp / Vector3 / Quaternion / Pose / Color helpers) factored out offrame_transforms_codec.cpp33dbe54Image,DepthImage,PointCloud; stale"PJ.CompressedVideo"reference inmedia_metadata_testupdated to"PJ.VideoFrame"301de99AssetVideobuiltin: file-backed video reference with playback metadata (sibling toVideoFrame's streamed-bitstream model)b84bd32Image.hpp→image.hpp, …); protos kept as-is since each file mirrors its message name0d29c19protobuf_wire_test_helpers.hppextracted; existingframe_transforms/image_annotationstests migrated to it (their golden-byte assertions are unchanged)b45b717docs/builtin_type.mdextended with per-type sections, classification rows, and conversion examples for the new typesTooling:
.clang-formatgained a second YAML document forLanguage: Proto(120-col)..pre-commit-config.yamlincludes.protoin clang-format and passes--allow-multiple-documentstocheck-yaml.Test results: 57/57 pass on Debug+ASAN (
./build.sh --debug && ./test.sh).Design notes
PJ::builtin_wireprimitives — nolibprotobufruntime dependency,no generated headers in public SDK. The pattern is sustainable
through the current type count (~3000 LOC across 11 codecs).
(
OccupancyGrid,CompressedPointCloud,Mesh3D,VideoFrame)follow the existing
Span<const uint8_t>+BufferAnchorpattern.New owned types (
SceneEntities,AssetVideo) own their fieldsdirectly. See updated
docs/builtin_type.md.AssetVideovsVideoFrame. Distinct families:AssetVideoisa one-per-topic file reference (LeRobot, MP4 datasets);
VideoFrameis one frame of a streamed bitstream with inter-frame state.
Test plan
./build.sh --debugclean under-Wall -Wextra -Werror+ ASAN./test.sh— 57/57 pass (48 pre-existing + 9 new codec tests)parseBuiltinObjectType()round-trips every enum entrykSchema<Type> == "PJ.<Type>"FrameTransformsandImageAnnotationskeep passing after the shared-helpers extraction🤖 Generated with Claude Code