issue with putobject persists even after forceAction=True

[zyzhang1130]

Hi, I still have problem putting small object (like apple) into a pickupable receptacle (like pot) even after I set forceAction=True. From direct observation the space inside the pot should be more than enough to contain the Apple. Same problem occurs when I tried to put apple even in a larger receptacle like sink. May I know what should I do?

Thank you.

[winthos]

forceAction = True does not force an object into another, all it does is remove the object restrictions that are in place by default (ie: by default Apple objects cannot be placed or randomly spawned inside drawers since that's not a natural place to find an Apple).

The space inside the pot might be misleading if only observing visually. The PutObject action has a limited number of positions to snap the Apple in place inside the pot, so it's likely that even though it looks like it could fit, the actual collision of the Apple or the Pot might prevent it from succeeding. Especially for smaller receptacle objects like pots, cups, and bowls, this behavior can persist. If you use the Drop action after moving the object over the receptacle with the MoveHand actions, you might have some better luck getting the objects to fall into the receptacle using physics, but the PutObject action specifically is meant to place objects into receptacles in a deterministic way, and in doing so it has less degrees of freedom in determining if an object will "fit" or not.

Apples should be small enough to fit into all sinks using the PutObject action, so if you could provide more details on exactly which scene, which sink, and which apple you are observing this behavior we can investigate.

[zyzhang1130]

Sure. It is the pot, apple and sink in FloorPlan1. I also tried to put the DishSponge and Egg in side the pot and succeeded. It seems it was indeed because of the apple size is too large, although I do feel that it would make sense if Apple and Tomato could be Put inside the pot as well (yea I tried Tomato also cannot).

Still the same scene, now regarding to the sink. Not only apple, but also DishSponge could not be Put into the sink. It is even stranger in DishSponge's case, because it was initialized in the sink to begin with.

Two more issues arised while I experimenting on Pot. As mentioned, I Put the Egg inside the Pot. And when I Pickup, the Pot together with Egg were picked up. Does it mean there is no way to get the Egg out of the Pot once it is inside? Or I can just Pickup Egg by specifying it id? The second one is, once I picked up Pot, Bowl and Pan from the CounterTop/TableTop, there is no way to put them down on the same table nor anywhere else anymore except on StoveBurner.

Thanks.

[winthos]

The pot in FloorPlan1 is one of the smaller pots in our library. Looking at the valid area where objects can be placed, I believe the reason the Apple, Tomato, and other objects are not being put in there is again due to the approximations used to attempt to snap held object into place in the pot. We cannot currently guarantee all small objects will be able to be placed in all instances of a receptacle category at this time.

As for the sink, the issue you are running into is that some Sinks are not guaranteed to be Receptacles. You will note in the documentation page here that Sinks are not always valid receptacles. We have split up the actionable areas of a sink into the Sink itself (see the Yellow highlighted portion in the example images) and the SinkBasin (see the green highlighted area in the examples)

This split is because some objects can be placed on the Sink geometry itself if there is room, like in the right example sink. Other Sinks don't have enough surface area to place anything on, as shown in the left example. All Sink objects have an associated SinkBasin object inside it, so you can explicitly place objects inside the sink's basin and not just ambiguously on the "sink." The details of how SinkBasin and Sinks are related can be found in the "Inherited" section of the object material properties documentation here. Note that objects like the Bathtub and BathtubBasin also work in this same way for this same reason.

If you put an egg on a plate or in a pot, you can pickup just the egg by itself by targeting it by objectId via the Pickup action. Any receptacle objects that are picked up will also automatically pickup all contained objects, but you can still grab just the contained objects as long as they fulfill the other requirements for the action (in range, visible and not occluded, etc).

Your last issue with not being able to place a picked up Pot/Bowl/Pan onto the CounterTop/Tabletop might be due to your field of view or perspective of the agent. In order for a held Pot/Bowl/Pan to be placed on a receptacle, the clear area on the CounterTop/Tabletop must be visible. If you had picked up a large object like this:

All the in-range Countertop area that is occluded by the object is not valid as a placement position. See below the approximate area that was occluded.

If you continue to have issues placing objects on receptacles that are visible clear, please double check the actionFinished and errorMessage action returns in the event object to see if they are throwing specific errors. Additionally, if you have more issues with object placement, providing example code for exactly what object you are trying to place on which receptacle at what coordinates (agent position, rotation, etc) I can look into it further.

[zyzhang1130]

Thank you for your detailed explanation. I solved the Sink issue by adding SinkBasin into my acceptable_receptacles.

As for the last question, I still couldn't put Bowl even the TableTop is so empty like this:

The agent-related info are:
'agent': {'name': 'agent', 'position': {'x': -0.2, 'y': 0.9009995, 'z': -1.3}, 'rotation': {'x': 0.0, 'y': 0.0, 'z': 0.0}, 'cameraHorizon': 30.0000038,

what I am executing is:
self.event = self.controller.step(dict(action=action_str, objectId=object_to_put['objectId'],
receptacleObjectId=interaction_obj['objectId'],forceAction=True, placeStationary=False))

this code worked if the object is other things like Apple, Egg ,etc:

[winthos]

I have a theory on why the bowl/pot/pan objects are failing to be placed. Based on your example code:

self.event = self.controller.step(dict(action=action_str, objectId=object_to_put['objectId'], receptacleObjectId=interaction_obj['objectId'],forceAction=True, placeStationary=False))

It seams like you are using the same objectId as both the receptacle to place the object and the object you are placing. My bet is what is happening is you are trying to place a held Bowl object inside the same Bowl object, and the action is failing because you can't place a bowl inside of itself. Double check that you are correctly trying to place a held Receptacle object onto the correct target receptacle. If all parameters are correct and the issue persists, a metadata dump of the attempted action might help debug this more.

[zyzhang1130]

You are right. The code did not account for the case where object_to_put is also a receptacle. Thank you for help identify that bug.