interfaces and algorithms for event based cameras, lidars, and actuators
Clone or download
Latest commit 302cb51 Sep 13, 2018

README.md

ch.ethz.idsc.retina+gokart Build Status

Sensor and actuator interfaces, Gokart software

The repository was developed with the following objectives in mind

  • interface sensors without loss of precision or temporal resolution
  • interface actuators of gokart using a protocol that is specific to the MicroAutoBox implementation

The code in the repository operates a heavy and fast robot that may endanger living creatures. We follow best practices and coding standards to protect from avoidable errors. See development_guidelines

usecase_gokart

Trajectory pursuit

planning_obstacles

Static obstacles

Features

  • interfaces to lidars Velodyne VLP-16, HDL-32E, Quanergy Mark8, HOKUYO URG-04LX-UG01
  • interfaces to event based camera Davis240C with lossless compression by 4x
  • lidar based localization
  • offline processing of log data

Architecture

We use LCM for message interchange. All messages are encoded using a single type BinaryBlob. The byte order of the binary data is little endian since the encoding is native on most architectures.

retina-repo

GOKART

gokart_operation

Actuation

PriorityModulePurposeRimoSteerLinmotMisc
HardwareSteerBatteryChargerprevent overcharging of batteryX
HardwareLinmotFireFighterprevent brake hardware damageX
EmergencyLinmotCoolingModuleno acceleration while temperature of linmot criticalX
EmergencyMiscEmergencyModuleno acceleration with steering battery lowX
EmergencySteerEmergencyModuleno acceleration with uncalibrated steeringX
EmergencyVlp16ClearanceModuleno acceleration towards a close obstacleX
EmergencyEmergencyBrakeProviderbrake maneuver based on obstacle in pathX
EmergencyLinmotTakeoverModuleswitch linmot to passive when driver pushes brake pedalX
EmergencyDeadManSwitchModulebrake if driver doesn't use joystick and gokart is above certain speedX
CalibrationSteerCalibrationProviderexecute steering calibration procedure, duration ~5[s]X
CalibrationLinmotCalibrationProviderexecute calibration of brake, duration ~4[s]X
CalibrationMiscIgnitionProviderACK of communication problem to microautobox by human operator, duration 0.3[s]X
ManualRimoTorqueJoystickModuletorque control by joystickX
ManualSteerJoystickModulesteering control by joystickX
ManualLinmotJoystickModulebraking by joystickX
TestingRimoComponentinteraction with motors in GUIX
TestingSteerComponentinteraction with steering in GUIX
TestingLinmotComponentinteraction with brake in GUIX
TestingMiscComponentinteraction with misc in GUIX
SafetyLinmotSafetyModuleno acceleration with brake disabledX
AutonomousPurePursuitRimocontrol of speed during pure pursuit trajectory followingX
AutonomousPurePursuitSteersteering during pure pursuitX
FallbackRimoPutFallbackzero torque on motorsX
FallbackSteerPutFallbackzero torque on steering columnX
FallbackLinmotPutFallbackmaintain operation in home positionX
FallbackMiscPutFallbacknormal operation, all LEDs offX

LIDAR

Velodyne VLP-16

  • point cloud visualization and localization with lidar video

Velodyne HDL-32E

  • 3D-point cloud visualization: see video

distance as 360[deg] panorama

velodyne distances

intensity as 360[deg] panorama

intensity

Quanergy Mark8

  • 3D-point cloud visualization: see video

HOKUYO URG-04LX-UG01

urg04lx

our code builds upon the urg_library-1.2.0

DVS

IniLabs DAVIS240C

Rolling shutter mode

05tram

04peds

00scene

Global shutter mode

dvs_2500

2.5[ms]

dvs_5000

5[ms]

Events only

dvs_noaps_1000

1[ms]

dvs_noaps_2500

2.5[ms]

dvs_noaps_5000

5[ms]

AEDAT 2.0, and AEDAT 3.1

  • parsing and visualization
  • conversion to text+png format as used by the Robotics and Perception Group at UZH
  • loss-less compression of DVS events by the factor of 2
  • compression of raw APS data by factor 8 (where the ADC values are reduced from 10 bit to 8 bit)

Device Settings

Quote from Luca/iniLabs:

  • Two parameters that are intended to control framerate: APS.Exposure and APS.FrameDelay
  • APS.RowSettle is used to tell the ADC how many cycles to delay before reading a pixel value, and due to the ADC we're using, it takes at least three cycles for the value of the current pixel to be output by the ADC, so an absolute minimum value there is 3. Better 5-8, to allow the value to settle. Indeed changing this affects the framerate, as it directly changes how much time you spend reading a pixel, but anything lower than 3 gets you the wrong pixel, and usually under 5-6 gives you degraded image quality.

We observed that in global shutter mode, during signal image capture the stream of events is suppressed. Whereas, in rolling shutter mode the events are more evenly distributed.

streaming DAT files

hdr

streaming DAVIS recordings

shapes_6dof

generating DVS from video sequence

cat_final

synthetic signal generation

synth2

synth1

Integration

Due to the rapid development of the code base, retina is not yet available as a maven artifact. Instead, download the project and run mvn install on your machine. Subsequently, you can use the project on your machine as

<dependency>
  <groupId>ch.ethz.idsc</groupId>
  <artifactId>retina</artifactId>
  <version>0.0.1</version>
</dependency>