Skip to content

[Bug Report] Very long 10+ minute load time #4151

New issue
New issue
Open
Open
[Bug Report] Very long 10+ minute load time#4151
@owenonline

Description

@owenonline
owenonline
opened on Dec 4, 2025

If you are submitting a bug report, please fill in the following details and use the tag [bug].

Describe the bug

I am experiencing incredibly long load times ranging from 1-20 minutes seemingly at random when using Isaac Sim. I've containerized my environment with this dockerfile:

FROM nvcr.io/nvidia/isaac-sim:5.0.0

# ------------------------------------------------------------
# Part 1: Core simulation stuff
# ------------------------------------------------------------

RUN apt-get update && apt-get install -y --no-install-recommends \
    git build-essential cmake unzip git-lfs wget \
  && rm -rf /var/lib/apt/lists/*

# Isaac Lab
RUN git clone https://github.com/isaac-sim/IsaacLab.git /IsaacLab && \
    cd /IsaacLab && \
    git checkout v2.2.0 && \
    ln -s /isaac-sim _isaac_sim && \
    TERM=xterm ./isaaclab.sh --install

# CycloneDDS (necessary to communicate with the simulated robot)
RUN git clone https://github.com/eclipse-cyclonedds/cyclonedds -b releases/0.10.x /cyclonedds && \
    cd /cyclonedds && mkdir build install && cd build && \
    cmake .. -DCMAKE_INSTALL_PREFIX=../install && \
    cmake --build . --target install
ENV CYCLONEDDS_HOME=/cyclonedds/install

# Unitree SDK 2.0
RUN git clone https://github.com/unitreerobotics/unitree_sdk2_python /unitree_sdk2_python && \
    cd /unitree_sdk2_python && \
    /isaac-sim/python.sh -m pip install -e .

# Unitree Sim environment
RUN git clone https://github.com/unitreerobotics/unitree_sim_isaaclab.git /home/code/unitree_sim_isaaclab && \
    cd /home/code/unitree_sim_isaaclab && \
    /isaac-sim/python.sh -m pip install -r requirements.txt

# Fetch USD assets
RUN cd /home/code/unitree_sim_isaaclab && \
    chmod +x fetch_assets.sh && \
    bash -lc ". fetch_assets.sh"

# ------------------------------------------------------------
# Part 2: VR Teleop stuff
# ------------------------------------------------------------

ARG COUNTRY="US"
ARG STATE="CA"
ARG LOCALITY="San Francisco"
ARG ORGANIZATION="University of California, Berkeley"
ARG ORGANIZATION_UNIT="BAIR"
ARG COMMON_NAME="Owen Burns"

ENV PATH="/root/miniconda3/bin:${PATH}"
ARG PATH="/root/miniconda3/bin:${PATH}"

# miniconda setup
RUN mkdir -p ~/miniconda3 && \
    wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh && \
    bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3 && \
    rm ~/miniconda3/miniconda.sh
RUN . ~/miniconda3/bin/activate
RUN conda init --all && \
    conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main && \
    conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/r && \
    conda create -n tv python=3.10 pinocchio=3.1.0 numpy=1.26.4 -c conda-forge

# XR Teleop stuff
RUN git clone https://github.com/unitreerobotics/xr_teleoperate.git /xr_teleoperate && \
    cd /xr_teleoperate && \
    git submodule update --init --depth 1 && \
    cd teleop/televuer && \
    ~/miniconda3/envs/tv/bin/pip install -e . && \
    openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout key.pem -out cert.pem -subj "/CN=${COUNTRY}/ST=${STATE}/L=${LOCALITY}/O=${ORGANIZATION}/OU=${ORGANIZATION_UNIT}/CN=${COMMON_NAME}" && \
    cd ../robot_control/dex-retargeting/ && \
    ~/miniconda3/envs/tv/bin/pip install -e . && \
    cd ../../../ && \
    ~/miniconda3/envs/tv/bin/pip install -r requirements.txt

# Install the unitree SDK in the conda environment as well
RUN cd /unitree_sdk2_python && \
    ~/miniconda3/envs/tv/bin/pip install -e .

# ------------------------------------------------------------
CMD [ "bash"]

Steps to reproduce

docker build -t unitree_sim_vr .
docker run --name isaac-sim --entrypoint bash -it --runtime=nvidia --gpus all -e "ACCEPT_EULA=Y" --rm --network=host \
    -e "PRIVACY_CONSENT=Y" \
    -v ~/docker/isaac-sim/cache/kit:/isaac-sim/kit/cache:rw \
    -v ~/docker/isaac-sim/cache/ov:/root/.cache/ov:rw \
    -v ~/docker/isaac-sim/cache/pip:/root/.cache/pip:rw \
    -v ~/docker/isaac-sim/cache/glcache:/root/.cache/nvidia/GLCache:rw \
    -v ~/docker/isaac-sim/cache/computecache:/root/.nv/ComputeCache:rw \
    -v ~/docker/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw \
    -v ~/docker/isaac-sim/data:/root/.local/share/ov/data:rw \
    -v ~/docker/isaac-sim/documents:/root/Documents:rw \
    -v /home/owenburns/workareas/teleop_data:/xr_teleoperate/teleop/utils/data \
    -v "$PWD":/workspace \
    unitree_sim_vr
conda deactivate
export PUBLIC_IP=$(curl -s ifconfig.me)
cd /home/code/unitree_sim_isaaclab
export PATH="/isaac-sim/kit/python/bin:$PATH"
/isaac-sim/python.sh sim_main.py --device cpu --enable_cameras --task Isaac-PickPlace-Cylinder-G129-Dex3-Joint --enable_dex3_dds --robot_type g129 --livestream 1

Is there any way to circumvent this very long load time, or reduce the probability of a long load time? Any help is greatly appreciated.

The key portion of the terminal log is:

[INFO]: Parsing configuration from: <class 'tasks.g1_tasks.press_knob_g1_29dof_dex3.press_knob_g1_29dof_dex3_joint_env_cfg.PressKnobG129DEX3JointEnvCfg'>

create environment...
Setting seed: 42
[INFO]: Base environment:
        Environment device    : cpu
        Environment seed      : 42
        Physics step-size     : 0.005
        Rendering step-size   : 0.01
        Environment step-size : 0.01
[INFO]: Time taken for scene creation : 1.043111 seconds
[INFO]: Scene manager:  <class InteractiveScene>
        Number of environments: 1
        Environment spacing   : 2.5
        Source prim name      : /World/envs/env_0
        Global prim paths     : []
        Replicate physics     : True
[INFO]: Starting the simulation. This may take a few seconds. Please wait...
2025-12-04T20:36:09Z [59,877ms] [Warning] [isaaclab.actuators.actuator_pd] The <ImplicitActuatorCfg> object has a value for 'effort_limit'. This parameter will be removed in the future. To set the effort limit, please use 'effort_limit_sim' instead.
2025-12-04T20:36:09Z [59,877ms] [Warning] [isaaclab.actuators.actuator_pd] The <ImplicitActuatorCfg> object has a value for 'velocity_limit'. Previously, although this value was specified, it was not getting used by implicit actuators. Since this parameter affects the simulation behavior, we continue to not use it. This parameter will be removed in the future. To set the velocity limit, please use 'velocity_limit_sim' instead.
2025-12-04T20:36:09Z [59,884ms] [Warning] [isaaclab.actuators.actuator_pd] The <ImplicitActuatorCfg> object has a value for 'effort_limit'. This parameter will be removed in the future. To set the effort limit, please use 'effort_limit_sim' instead.
2025-12-04T20:36:09Z [59,884ms] [Warning] [isaaclab.actuators.actuator_pd] The <ImplicitActuatorCfg> object has a value for 'velocity_limit'. Previously, although this value was specified, it was not getting used by implicit actuators. Since this parameter affects the simulation behavior, we continue to not use it. This parameter will be removed in the future. To set the velocity limit, please use 'velocity_limit_sim' instead.
2025-12-04T20:36:09Z [60,167ms] [Warning] [usdrt.population.plugin] [UsdNoticeHandler] Unhandled attribute type VtArray<std::string> (prim attribute: omni:rtx:material:db:flattener:transmittance_color)
2025-12-04T20:36:09Z [60,167ms] [Warning] [usdrt.population.plugin] [UsdNoticeHandler] Unhandled attribute type VtArray<std::string> (prim attribute: omni:rtx:material:db:flattener:ior_constant)
2025-12-04T20:36:09Z [60,167ms] [Warning] [usdrt.population.plugin] [UsdNoticeHandler] Unhandled attribute type VtArray<std::string> (prim attribute: omni:rtx:material:db:flattener:transmittance_color)
2025-12-04T20:36:09Z [60,169ms] [Warning] [usdrt.population.plugin] [UsdNoticeHandler] Unhandled attribute type VtArray<std::string> (prim attribute: omni:rtx:material:db:flattener:ior_constant)
2025-12-04T20:36:09Z [60,169ms] [Warning] [usdrt.population.plugin] [UsdNoticeHandler] Unhandled attribute type VtArray<std::string> (prim attribute: omni:rtx:material:db:flattener:reflection_roughness_constant)
2025-12-04T20:36:09Z [60,169ms] [Warning] [usdrt.population.plugin] [UsdNoticeHandler] Unhandled attribute type VtArray<std::string> (prim attribute: omni:rtx:material:db:flattener:transmittance_color)
2025-12-04T20:36:09Z [60,170ms] [Warning] [usdrt.population.plugin] [UsdNoticeHandler] Unhandled attribute type VtArray<std::string> (prim attribute: omni:rtx:material:db:flattener:transmittance_color)
2025-12-04T20:36:09Z [60,170ms] [Warning] [usdrt.population.plugin] [UsdNoticeHandler] Unhandled attribute type VtArray<std::string> (prim attribute: omni:rtx:material:db:flattener:reflection_roughness_constant)
2025-12-04T20:36:09Z [60,170ms] [Warning] [usdrt.population.plugin] [UsdNoticeHandler] Unhandled attribute type VtArray<std::string> (prim attribute: omni:rtx:material:db:flattener:reflection_roughness_constant)
2025-12-04T20:36:09Z [60,170ms] [Warning] [usdrt.population.plugin] [UsdNoticeHandler] Unhandled attribute type VtArray<std::string> (prim attribute: omni:rtx:material:db:flattener:reflection_roughness_constant)
2025-12-04T20:36:09Z [60,171ms] [Warning] [usdrt.population.plugin] [UsdNoticeHandler] Unhandled attribute type VtArray<std::string> (prim attribute: omni:rtx:material:db:flattener:ior_constant)
2025-12-04T20:36:09Z [60,171ms] [Warning] [usdrt.population.plugin] [UsdNoticeHandler] Unhandled attribute type VtArray<std::string> (prim attribute: omni:rtx:material:db:flattener:ior_constant)
2025-12-04T20:36:09Z [60,220ms] [Warning] [usdrt.hydra.fabric_scene_delegate.plugin] using high frequency span with attrs is disabled
2025-12-04T20:36:09Z [60,266ms] [Warning] [usdrt.hydra.fabric_scene_delegate.plugin] using high frequency span with attrs is disabled
2025-12-04T20:36:09Z [60,267ms] [Warning] [omni.fabric.plugin] getAttributeCount called on non-existent path /World/envs/env_0/Robot/right_ankle_roll_link/collisions/mesh_3
2025-12-04T20:36:09Z [60,267ms] [Warning] [omni.fabric.plugin] getTypes called on non-existent path /World/envs/env_0/Robot/right_ankle_roll_link/collisions/mesh_3
2025-12-04T20:46:15Z [665,978ms] [Warning] [rtx.postprocessing.plugin] DLSS increasing input dimensions: Render resolution of (371, 278) is below minimal input resolution of 300.
2025-12-04T20:46:15Z [666,348ms] [Warning] [omni.syntheticdata.plugin] OgnSdPostRenderVarToHost : rendervar copy from texture directly to host buffer is counter-performant. Please use copy from texture to device buffer first.
[INFO]: Time taken for simulation start : 607.622739 seconds
[INFO] Command Manager:  <CommandManager> contains 0 active terms.

Notice the simulation start time of 607 seconds. This behavior is consistent, even though all of the assets involved are already present locally and no USD files are downloaded.

System Info

Describe the characteristic of your environment:

  • Commit: v2.2.0
  • Isaac Sim Version: 5.0.0
  • OS: Ubuntu 22.04
  • GPU: NVIDIA RTX 6000 Ada
  • CUDA: 12.8
  • GPU Driver: 570.195.03

Additional context

Add any other context about the problem here.

Checklist

  • I have checked that there is no similar issue in the repo (required)
  • I have checked that the issue is not in running Isaac Sim itself and is related to the repo (see issue opened in isaac sim repo, was told to re-open issue here)

Acceptance Criteria

Add the criteria for which this task is considered done. If not known at issue creation time, you can add this once the issue is assigned.

  • Criteria 1: Resolution for long load times, preferably under 1 minute. 10+ minutes destroys our iteration speed as it means only 3 code changes (no matter how small) can realistically be tested per hour.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions