`get_rgba()` returned unexpected array

[Tommy861]

Description

While running my script in Isaac Sim, I encountered a crash when calling the get_rgba() function of the camera sensor. The error indicates that the returned array is 1-dimensional, while the code expects a 3-dimensional array. This suggests that get_rgba() might occasionally return a non-expected shape.

From the definition of get_rgba(), the return value of this function is not a one-dimensional array.
https://docs.isaacsim.omniverse.nvidia.com/latest/py/source/extensions/isaacsim.sensors.camera/docs/index.html#isaacsim.sensors.camera.Camera.get_rgba

Here is my script that produces the crash:

from isaacsim import SimulationApp
simulation_app = SimulationApp({"headless": True})

import isaacsim.core.utils.numpy.rotations as rot_utils
import numpy as np
from isaacsim.core.api import World
from isaacsim.core.api.objects import DynamicCuboid
from isaacsim.sensors.camera import Camera
from PIL import Image, ImageDraw

width, height = 1920, 1200
camera_matrix = [[958.8, 0.0, 957.8], [0.0, 956.7, 589.5], [0.0, 0.0, 1.0]]
distortion_coefficients = [0.14, -0.03, -0.0002, -0.00003, 0.009, 0.5, -0.07, 0.017]
pixel_size = 3
f_stop = 1.8
focus_distance = 1.5
world = World(stage_units_in_meters=1.0)
world.scene.add_default_ground_plane()

cube_1 = world.scene.add(
    DynamicCuboid(
        prim_path="/new_cube_1",
        name="cube_1",
        position=np.array([0, 0, 0.5]),
        scale=np.array([1.0, 1.0, 1.0]),
        size=1.0,
        color=np.array([255, 0, 0]),
    )
)

cube_2 = world.scene.add(
    DynamicCuboid(
        prim_path="/new_cube_2",
        name="cube_2",
        position=np.array([2, 0, 0.5]),
        scale=np.array([1.0, 1.0, 1.0]),
        size=1.0,
        color=np.array([0, 255, 0]),
    )
)

cube_3 = world.scene.add(
    DynamicCuboid(
        prim_path="/new_cube_3",
        name="cube_3",
        position=np.array([0, 4, 1]),
        scale=np.array([2.0, 2.0, 2.0]),
        size=1.0,
        color=np.array([0, 0, 255]),
    )
)

camera = Camera(
    prim_path="/World/camera",
    position=np.array([0.0, 0.0, 2.0]),  
    frequency=30,
    resolution=(width, height),
    orientation=rot_utils.euler_angles_to_quats(np.array([0, 90, 0]), degrees=True),
)

world.reset()
camera.initialize()
((fx, _, cx), (_, fy, cy), (_, _, _)) = camera_matrix  
horizontal_aperture = pixel_size * width * 1e-6  
vertical_aperture = pixel_size * height * 1e-6  
focal_length_x = pixel_size * fx * 1e-6  
focal_length_y = pixel_size * fy * 1e-6  
focal_length = (focal_length_x + focal_length_y) / 2  
camera.set_focal_length(focal_length)
camera.set_focus_distance(focus_distance)
camera.set_lens_aperture(f_stop)
camera.set_horizontal_aperture(horizontal_aperture)
camera.set_vertical_aperture(vertical_aperture)
camera.set_clipping_range(0.05, 1.0e5)
camera.set_opencv_pinhole_properties(cx=cx, cy=cy, fx=fx, fy=fy, pinhole=distortion_coefficients)
for i in range(100):
    world.step(render=True)
    if world.is_done():
        break
camera.get_current_frame()
img = Image.fromarray(camera.get_rgba()[:, :, :3])

simulation_app.close()

Isaac Sim version

5.1.0

Operating System (OS)

Windows 10

GPU Name

RTX 4060 Laptop GPU

GPU Driver and CUDA versions

Driver 577.00, CUDA 12.9

Logs

2025-10-10T14:06:06Z [29,591ms] [Error] [omni.kit.app._impl] [py stderr]: Traceback (most recent call last):
2025-10-10T14:06:06Z [29,592ms] [Error] [omni.kit.app._impl] [py stderr]:   File "C:\IsaacSim\_build\windows-x86_64\release\api_isaacsim.sensors.camera_camera_opencv_pinhole_mutated_by_add_one_method_20250926_124738.py", line 93, in <module>
2025-10-10T14:06:06Z [29,593ms] [Error] [omni.kit.app._impl] [py stderr]:     img = Image.fromarray(camera.get_rgba()[:, :, :3])
2025-10-10T14:06:06Z [29,593ms] [Error] [omni.kit.app._impl] [py stderr]:                           ~~~~~~~~~~~~~~~~~^^^^^^^^^^
2025-10-10T14:06:06Z [29,594ms] [Error] [omni.kit.app._impl] [py stderr]: IndexError: too many indices for array: array is 1-dimensional, but 3 were indexed 

Additional information

No response

[vick-yu]

Thank you for sharing the detailed report and script.

After reviewing, I found that the early exit condition

if world.is_done():
    break

may lead to the simulation loop terminating before valid camera data is available, which in turn causes get_rgba() to return an unexpected array shape.

When I removed this early exit snippet, the crash was resolved and the camera output was as expected.
To avoid this issue, I recommend ensuring the simulation runs for the required number of steps to allow all sensors, including the camera, to initialize and produce valid data.

[vick-yu]

To ensure you receive valid camera output, it's useful to step the simulation for several frames before checking image data. If you include a check for sensor output (like below), you’ll see valid data after a few steps:

for i in range(100):
    world.step(render=True)
    print(f"Step {i}")
    data = camera.get_rgba()
    if len(data) > 0:
        print(f"  data shape: {data.shape}, received data after {i} steps")
        break
    print(f"  data: {data}")
    # if world.is_done():
    #     break

camera.get_current_frame()
img = Image.fromarray(camera.get_rgba()[:, :, :3])

In my runs, valid data after the renderer's warmup phase was typically available after ~5 steps, with the returned array correctly shaped as an image.

[Tommy861]

To ensure you receive valid camera output, it's useful to step the simulation for several frames before checking image data. If you include a check for sensor output (like below), you’ll see valid data after a few steps:

for i in range(100):
world.step(render=True)
print(f"Step {i}")
data = camera.get_rgba()
if len(data) > 0:
print(f" data shape: {data.shape}, received data after {i} steps")
break
print(f" data: {data}")
# if world.is_done():
# break

camera.get_current_frame()
img = Image.fromarray(camera.get_rgba()[:, :, :3])
In my runs, valid data after the renderer's warmup phase was typically available after ~5 steps, with the returned array correctly shaped as an image.

@vick-yu Thank you very much for your detailed explanation! Since the get_rgba function may return a vector with unexpected dimensions (inconsistent with the documentation), it is suggested that a note be added to the documentation to inform developers about this behavior, including the need to wait for valid data after the renderer’s warm-up phase.

[vick-yu]

Thank you for reporting this and for your suggestion regarding the documentation. An internal ticket has been created to address the get_rgba() documentation note so that future users are aware of potential output shape issues before renderer warmup.
Closing this issue as it is now tracked internally.