Can we change the feature of the objects in scenes ?

[liuxz-cs]

I want to know that if we can change the texture or color or other feature of the objects in scenes. If can, how can we do that ?
And also, I want to know that if we can add some objects into scenes. If can, how can we do that ?

[mattdeitke]

can change the texture or color or other feature of the objects in scenes. If can, how can we do that ?

Textures cannot currently be changed. For colors, there is a currently undocumented function called ChangeColorOfMaterials that could work well. But, it doesn't currently allow you to reset the scenes to their original colors after initially changing randomizing a scene's colors.

Here's a brief example:

which can be reproduced with

from ai2thor.controller import Controller
import matplotlib.pyplot as plt

c = Controller()
scenes = [f'FloorPlan{i}' for i in ['28', '201', '301']]
fig, axs = plt.subplots(3, 3, figsize=(10, 10))
for i, scene in enumerate(scenes):
    c.reset(scene)
    for j in range(3):
        axs[i,j].imshow(c.last_event.frame)
        axs[i,j].set_xticks([])
        axs[i,j].set_yticks([])

        # change colors
        c.step('ChangeColorOfMaterials')

For even crazier effects, it might be worth incorporating imgaug or other common augmentations within torchvision's transforms submodule.

I want to know that if we can add some objects into scenes.

Currently, there is support for duplicating existing objects in the scene (or alternatively removing objects currently in the scene) using InitialRandomSpawn, documented here. But, adding in different objects across scenes is not yet supported.

[liuxz-cs]

can change the texture or color or other feature of the objects in scenes. If can, how can we do that ?

Textures cannot currently be changed. For colors, there is a currently undocumented function called ChangeColorOfMaterials that could work well. But, it doesn't currently allow you to reset the scenes to their original colors after initially changing randomizing a scene's colors.

Here's a brief example:

which can be reproduced with

from ai2thor.controller import Controller
import matplotlib.pyplot as plt

c = Controller()
scenes = [f'FloorPlan{i}' for i in ['28', '201', '301']]
fig, axs = plt.subplots(3, 3, figsize=(10, 10))
for i, scene in enumerate(scenes):
    c.reset(scene)
    for j in range(3):
        axs[i,j].imshow(c.last_event.frame)
        axs[i,j].set_xticks([])
        axs[i,j].set_yticks([])

        # change colors
        c.step('ChangeColorOfMaterials')

For even crazier effects, it might be worth incorporating imgaug or other common augmentations within torchvision's transforms submodule.

I want to know that if we can add some objects into scenes.

Currently, there is support for duplicating existing objects in the scene (or alternatively removing objects currently in the scene) using InitialRandomSpawn, documented here. But, adding in different objects across scenes is not yet supported.

So, when we use the function ChangeColorOfMaterials, the color is changed randomly ? Can we choose which color to change by this function?

[mattdeitke]

So, when we use the function ChangeColorOfMaterials, the color is changed randomly ?

Correct.

Can we choose which color to change by this function?

At the moment, no. But, I suppose using instance segmentation masks, one could mask each object individually and then change the hue such that the color changes to some desired color. Although this would be decently expensive from a time perspective.

[liuxz-cs]

So, when we use the function ChangeColorOfMaterials, the color is changed randomly ?

Correct.

Can we choose which color to change by this function?

At the moment, no. But, I suppose using instance segmentation masks, one could mask each object individually and then change the hue such that the color changes to some desired color. Although this would be decently expensive from a time perspective.

Okay, thanks a lot.