FieldGen

Large-scale diverse data underpin robust manipulation, yet existing pipelines trade scale, diversity, and quality: simulation scales but leaves sim-to-real gaps; teleoperation is precise but costly and behaviorally narrow. FieldGen is a field-guided trajectory generation & data scaling framework.

Features

• Trajectory types: Bezier / Cone (structured approach path)
• Endpoint guided: local Z axis reverse assist in curve construction
• Non‑uniform sampling: denser near endpoint (stabler learning)
• Length alignment: unified by chunk_size (pad if short, truncate if long)
• Reward mode: --reward multiplies samples with multi‑radius endpoint perturbation + normalized distance reward
• Multi‑task batching: iterate multiple task folders
• Standardized HDF5 layout: action / state / eef / effector / reward
• Rich visualization: 3D scatter, multi‑view density HTML; trajectory + orientation arrows
• Centralized YAML configuration
• Minimal algorithms: easy to plug in new trajectory generators

Repository Layout (key files)

fieldgen/
   generate.py              # main generation entry
   requirements.txt         # Python dependencies
   config/
      config.yaml           # main configuration
   utils/
      bezier_util.py        # Bezier curve utilities
      cone_util.py          # Cone half‑cycloid utilities
      rpy_util.py           # RPY / quaternion interpolation helpers
      visualize_points.py   # point & trajectory visualization
   scripts/                 # data conversion, splits, HDF5 inspection
   tests/                   # demo style tests

Installation

Python >= 3.10 recommended.

git clone <repo-url>
cd fieldgen
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Optional interactive visualization:

pip install plotly

Base dependencies (see requirements.txt): numpy, scipy, pyyaml, h5py, Pillow, tqdm, matplotlib (+ optional plotly).

Configuration (config/config.yaml)

generate section (selected fields):

Field	Meaning
curve_type	bezier or cone trajectory type
output_path	Root directory for generated episode outputs
chunk_size	Unified trajectory length T
beta	Secondary sampling ratio control (point count estimation)
endpoint_random_radius	Max perturbation radius in reward mode
multiplier	Augmentation factor per original point (reward mode)
reward_target_min / max	Clipping interval for normalized reward

tasks section:

• key: task name (e.g. pick0)
• path: source directory (must contain sample_points.h5, optional camera/*/*.jpg)
• max_trajectories: limit number processed (null = all)

Example:

generate:
   curve_type: cone
   output_path: /path/to/output
   chunk_size: 35
   beta: 0.0025
   endpoint_random_radius: 0.1
   multiplier: 10
   reward_target_min: 0.0
   reward_target_max: 1.0

tasks:
   pick0:
      path: /path/to/episode0
      max_trajectories: null
   pick1:
      path: /path/to/episode1
      max_trajectories: null

Input Data Requirements

Each task folder must include:

<task_path>/
   sample_points.h5
   camera/
       0/*.jpg
       1/*.jpg   # subdirectories organized by eef index

Mandatory datasets inside sample_points.h5:

• state/eef/position : shape (N, >=12); columns [6:9] starting xyz, [9:12] starting rpy
• endpoint : shape (>=12,); [6:9] final xyz, [9:12] final rpy

Output Format

Each generated episodeK/aligned_joints.h5 contains:

timestamps : (T,)
action/eef/position : (T, 12)
state/eef/position  : (T, 12)
action/effector/position : (T, 2)   # normalized (÷90)
state/effector/position  : (T, 2)
reward/value : scalar or (T,)       # only in --reward mode

Trajectory concatenation ordering: [left_xyz(3), left_rpy(3), right_xyz(3), right_rpy(3)] = 12. Images: All relevant .jpg copied to episodeK/camera/0/.

Usage

Deterministic endpoint mode:

python generate.py --config config/config.yaml

Reward augmentation mode (random endpoints + reward writing):

python generate.py --config config/config.yaml --reward

Data Collection

FieldGen supports two complementary data acquisition modes for building large, diverse approach (pre‑manipulation) datasets:

1. Automated Workspace Filling: Use the helper script (see document/data_collect_notice.md) to auto‑calibrate a reachable workspace from a few existing episodes, then generate a dense set of coverage points via 3D Z‑order (Morton) or Hilbert space‑filling curves. Each point optionally receives a synthesized sinusoidal RPY perturbation sequence to enrich orientation diversity.
2. Teleoperation Traces: Manually drive the robot to collect longer (≈1–2 min) free‑space approach trajectories while lightly rotating the wrist/gripper and keeping the target object centered in view. This yields high‑quality, contact‑proximal behavior diversity where automated methods may underperform.

Full details are documented here: ▶ Data Collection Guide

Reward Mechanism

In reward mode:

1. Read original endpoint original_endpoint_pos
2. Stratified sample perturbation distances within [0, endpoint_random_radius * (1 - reward_target_min)]
3. Sample random direction vectors and perturb endpoint
4. Normalize distance: reward = 1 - dist / R_radius, clip to [reward_target_min, reward_target_max]
5. Generate one trajectory per perturbed endpoint + associated reward/value

Extensible ideas: include orientation deviation, smoothness, obstacle clearance, etc.

Trajectory Algorithms

Quadratic Bezier Curve

• Control points: start, end, projection adjustment using (end + direct) direction
• Arc length estimate + power resampling (power > 1 densifies tail; <1 densifies head)
• Degenerate case (direct≈0): fall back to midpoint control

In-Cone Half‑Cycloid

• If start inside cone: direct half‑cycloid to apex
• If start outside cone: axial line inwards + half‑cycloid segment
• Allocate sample counts by relative estimated length of segments

RPY Interpolation

• Same‑hemisphere quaternion handling avoids long rotations
• Quadratic ease‑out for monotonically decreasing angular velocity
• Accumulate axis‑angle increments for sequence

Visualization

python utils/visualize_points.py

Generates:

• html/point_distribution_3d_<task>.html
• html/point_distribution_2d_density_<task>.html Open in your browser.

FAQ

Question	Suggestion
Trajectories too short (heavy padding)	Tune beta or reduce chunk_size
Not dense enough near endpoint	Increase internal Bezier/Cone power value
Reward variance too small	Increase endpoint_random_radius or lower reward_target_min
Add new trajectory type	Create utils/xxx_curve.py and extend factory in generator
Image copy slows processing	Limit per episode images or disable copying

License

This project is licensed under the MIT License – see the LICENSE file for details.

Contributing

PRs and issues are welcome:

• New trajectory generation algorithms & sampling strategies
• Data conversion & format enhancements
• Robust unit tests & CI integration

Citation (Placeholder)

If you use FieldGen in academic work, please cite it. (BibTeX will be added when a paper is published.)

---

If FieldGen helps your research or project, consider starring the repository. Thank you!