Main components and functions

Components

The SGG is broken into several components:

Main components

• The Scene Graph Generator (SGG) carla_sgg/sgg.py
  • Generates the "base SG" - the most semantically rich scene graph
• The Scene Graph Abstractor (SGA) carla_sgg/sgg_abstractor.py
  • Functions for abstracting the base SG into forms more applicable to the given task. See the Abstractions wiki page for more information about the different abstractions available.

Helper components

• utils.py
  • Helper functions for working with different objects
• viz.py
  • Functions to plot/visualize scene graphs
• actors.py
  • Mappings between different CARLA actor types and their attributes within the SG

Methods

The Scene Graph Generator (SGG)

The SGG lives in carla_sgg/sgg.py and declares a type SGG that must be instantiated for the given simulation episode. Then, at each time step, the SGG can be invoked to generate a new scene graph based on the current state of the simulation. See this guide for more information about setting up a minimum viable example.

The Constructor

The SGG takes in a reference to the CARLA client object being used for the simulation as well as the ID of the ego vehicle. The ego vehicle, also called the hero vehicle in some documentation, is the vehicle under examination. All scene graphs contain an ego vehicle at their core, and implicitly everything is defined relative to the ego vehicle. Below is a minimum viable usage of the constructor.

    # setup the CARLA client
    client = carla.Client('localhost', 2000)
    # This is longer than typical - the client timeout must wait for the SGG indexing to complete
    client.set_timeout(120.0)

    # load the environment
    world = client.load_world('Town10_Opt')  # set town string to load a different map

    # it is recommended to run the simulation in synchronous mode when generating scene graphs
    # to ensure alignment between the scene graph and other sensor data
    settings = world.get_settings()
    settings.synchronous_mode = True
    settings.fixed_delta_seconds = 0.05
    world.apply_settings(settings)
    world = client.reload_world(False)

    # create the ego vehicle
    blueprint_library = world.get_blueprint_library()
    ego_bp = blueprint_library.find('vehicle.tesla.model3')  # change the blueprint id for a different vehicle type
    spawn_points = world.get_map().get_spawn_points()
    ego = world.spawn_actor(ego_bp, spawn_points[0])

    # Setup the SGG - this call will take several seconds while the SGG indexes the world and performs initial bookkeeping
    sgg = SGG(client, ego.id)

Generating the SG

The method generate_graph_for_frame generates the SG based on the current simulation state. This method takes in two parameters: the world snapshot and the ego vehicle control commands. The control commands are optional and are used to enrich the SG based on the commanded action for the ego vehicle to contrast with the actual action taken by the vehicle; this is useful for several types of analysis. This can either be a dictionary as shown below, or an instance of carla.VehicleControl.

        for frame_number in range(100):
            # get the simulation state world snapshot
            frame = world.tick()
            # can either be a dictionary as below, or carla.VehicleControl
            ego_control = {
                'throttle': 5.0,
                'steer': -1.0
            }
            sg = sgg.generate_graph_for_frame(frame, ego_control)