Rotation angle errors for PythonAPI

[onorabil]

Issue: listener yields wrong rotation values/images from PythonAPI
Version: 0.9.10, Ubuntu 18.04; seems to affect older versions as well
Expected behavior: set_transform pose should match the output pose and image, for example

0
[input]xyzPitchYawRoll:  0.0 0.0 50.0 -90.0 0.0 80.0
[output]xyzPitchYawRoll: 0.0 0.0 50.0 -90.0 0.0 80.0
1
[input]xyzPitchYawRoll:  0.0 0.0 50.0 -90.0 0.0 70.0
[output]xyzPitchYawRoll: 0.0 0.0 50.0 -90.0 0.0 70.0
2
[input]xyzPitchYawRoll:  0.0 0.0 50.0 -90.0 0.0 50.0
[output]xyzPitchYawRoll: 0.0 0.0 50.0 -90.0 0.0 50.0

Experienced behavior

0
[input]xyzPitchYawRoll:  0.0 0.0 50.0 -90.0 0.0 80.0
[output]xyzPitchYawRoll: 0.0 0.0 50.0 -90.0 9.462335586547852 70.53760528564453
1
[input]xyzPitchYawRoll:  0.0 0.0 50.0 -90.0 0.0 70.0
[output]xyzPitchYawRoll: 0.0 0.0 50.0 -90.0 -7.125022888183594 77.12506866455078
2
[input]xyzPitchYawRoll:  0.0 0.0 50.0 -90.0 0.0 50.0
[output]xyzPitchYawRoll: 0.0 0.0 50.0 -90.0 -90.0 139.99990844726562

Code to reproduce the numbers above

"""Reproduce transform wrong location bug @ listener"""

import glob
import os
import sys
try:
    sys.path.append(glob.glob('../carla/dist/carla-*%d.%d-%s.egg' % (
        sys.version_info.major,
        sys.version_info.minor,
        'win-amd64' if os.name == 'nt' else 'linux-x86_64'))[0])
except IndexError:
    pass

import carla

import time

client = carla.Client('127.0.0.1', 2000)

#world = client.load_world('Town04')


OUTPUT_FOLDER = 'output'

if not os.path.exists(OUTPUT_FOLDER):
    os.makedirs(OUTPUT_FOLDER)

world = client.get_world()



OUTPUT_WIDTH = '854'
OUTPUT_HEIGHT = '854'
OUTPUT_FOV = '84'

OUTPUT_WIDTH_FLOAT = int(float(OUTPUT_WIDTH))
OUTPUT_HEIGHT_FLOAT = int(float(OUTPUT_HEIGHT))


def process_data(data, sensor_type):
    image_name = '%s/image_%s_x_%f_y_%f_z_%f_pitch_%f_yaw_%f_roll_%f.png' % (OUTPUT_FOLDER, \
        sensor_type, data.transform.location.x, data.transform.location.y, \
        data.transform.location.z, data.transform.rotation.pitch, data.transform.rotation.yaw, \
        data.transform.rotation.roll)
    if os.path.exists(image_name):
        return
    print('[output]xyzPitchYawRoll:', data.transform.location.x, data.transform.location.y, \
        data.transform.location.z, data.transform.rotation.pitch, data.transform.rotation.yaw, \
        data.transform.rotation.roll)
    data.save_to_disk(image_name)


# rgb
blueprint_rgb = world.get_blueprint_library().find('sensor.camera.' + 'rgb')
blueprint_rgb.set_attribute('image_size_x', OUTPUT_WIDTH)
blueprint_rgb.set_attribute('image_size_y', OUTPUT_HEIGHT)
blueprint_rgb.set_attribute('fov', OUTPUT_FOV)
blueprint_rgb.set_attribute('sensor_tick', '0.0')
transform = carla.Transform(carla.Location(x=0.8, z=20))
sensor_rgb = world.spawn_actor(blueprint_rgb, transform, attach_to=None)
sensor_rgb.listen(lambda data: process_data(data, 'rgb'))

# SAMPLE POINTS

points = \
"""x 0 y 0 z 50 yaw 0 pitch -90 roll 80
x 0 y 0 z 50 yaw 0 pitch -90 roll 70
x 0 y 0 z 50 yaw 0 pitch -90 roll 50
"""

lines = points.splitlines()
final_x =[]
final_y = []
final_z = []
final_rolls = []
final_yaws = []
final_pitches = []
for line in lines:
    splits = line.split(' ')
    final_x.append(float(splits[1]))
    final_y.append(float(splits[3]))
    final_z.append(float(splits[5]))
    final_yaws.append(float(splits[7]))
    final_pitches.append(float(splits[9]))
    final_rolls.append(float(splits[11]))


for idx_image in range(len(final_x)):
    print(idx_image)

    current_x = final_x[idx_image]
    current_y = final_y[idx_image]
    current_z = final_z[idx_image]
    current_yaw = final_yaws[idx_image]
    current_pitch = final_pitches[idx_image]
    current_roll = final_rolls[idx_image]

    print('[input]xyzPitchYawRoll:', current_x, current_y, current_z, current_pitch, current_yaw, current_roll)

    transform = carla.Transform(carla.Location(x=float(current_x), y=float(current_y), z=float(current_z)), \
        carla.Rotation(pitch=float(current_pitch), yaw=float(current_yaw), roll=float(current_roll)))
    transformRotationOnly = carla.Transform(carla.Location(x=float(0), y=float(0), z=float(0)), \
        carla.Rotation(pitch=float(current_pitch), yaw=float(current_yaw), roll=float(current_roll)))
    #print('[input]rotation matrix:', transformRotationOnly.get_matrix())
    #print('[input]angles from rotation matrix:', transformRotationOnly.rotation.pitch, transformRotationOnly.rotation.yaw, transformRotationOnly.rotation.roll)
    sensor_rgb.set_transform(transform)
    time.sleep(2) 
    sensor_rgb.set_transform(transform)
    time.sleep(2) 

Detailed steps to reproduce

wget https://carla-releases.s3.eu-west-3.amazonaws.com/Linux/CARLA_0.9.10.tar.gz
tar -xvf CARLA_0.9.10.tar.gz
sh ./CarlaUE4.sh
# new terminal, copy & run the code above from PythonAPI/examples

Comments

Originally, the script was used in synchronous mode. We increased the fixed_delta_seconds parameter hoping it is related to a short time window to set the transform, but we were wrong. The snippet above deals away with the time step issues, even though it is inefficient (process_data called multiple times for the same pose). Setting the transform twice / sleeping is just for the sake of this experiment.

For some values it works fine (~e-05 errors). We need accurate relative camera pose as ground truth.

Am I doing something wrong?

[corkyw10]

@Axel1092 would you be able help with this please?

[Axel1092]

Hi @onorabil, this differences in rotation angles are to be expected as they are the result of internal processing by unreal. When you set the actor rotation, unreal takes the angle values and normalizes them according to its internal angle boundaries. In the end you get the same rotation (or very similar) as your input with different angle value combination.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.