Issue in generating scenarios that make sense and are as realistic as possible

[basteran]

Hello everyone, thank you for the great work.

I've been using this repository for a while now trying to generate some scenarios that make sense and are as realistic as possible. Currently, I am facing some little problems, such as:

• no wc in the bathroom
• microwave or pans on the floor in the kitchen
• wardrobes or shelves in front of the windows
• no isle in the kitchen although I manually specified this constraint to ChatGPT in the prompt
• huge, but quite empty, bedrooms
• missing specified objects (e.g. a book on the table, or the remote controller in the sofa)

Do you think this is to be expected? Or is it my problem (maybe the prompt is not that stable and my edits lead to unstable results?)?
I even tried to generate some variations, but I see no improvement besides the aspect and the assets.

Could you help me out?
Thanks again.

[YueYANG1996]

I encountered some of the problems you raised. Here are some suggestions for each of them:

• No WC in the bathroom: I feel this is a rare case. It may happen, but usually, the toilet is the most important object in the bathroom, and the holodeck should be able to generate it first.

• microwave or pans on the floor in the kitchen: I also noticed this microwave issue. One thing you can do is to add another check for the object list here You can have another LLM to verify if this plan is reasonable and you can fix some mistakes here, e.g., the microwave should be on the top of table not on the floor.

• wardrobes or shelves in front of the windows: you can change this variable to True. In this case, we will treat windows as objects, so the object placement will consider the collision with windows.

• no isle in the kitchen although I manually specified this constraint to ChatGPT in the prompt: This is due to the lack of good isle assets in our subset of objaverse, but I think most of the time, if the kitchen is large enough, there should be an isle or kitchen counter.

• huge, but quite empty, bedrooms: you can specify the size of the room in the prompt, for example, "5m x 5m". Or if you want more objects in the room, you can increase this value. This value controls the capacity ratio of objects for each room.

• missing specified objects (e.g., a book on the table or the remote controller on the sofa): adding them in the prompt can help; however, since scene generation is a long and complicated process, it's hard to ensure every detail can be satisfied.

[basteran]

Thanks for the exhaustive answer!

missing specified objects (e.g., a book on the table or the remote controller on the sofa): adding them in the prompt can help; however, since scene generation is a long and complicated process, it's hard to ensure every detail can be satisfied.

is there a way to force (or at least to control) the generation and placement of objects? I am interested in creating some realistic scenarios with a few constraints, but I feel like most of the time the constraints I am adding are not satisfied. I have a few examples:

• a book on the dining table
• a chair near the sofa
• a bottle far from the kitchen
• a blanket (wherever in the environment)

I would like to make sure that at least these constraints are met. I can also modify the code if necessary, but I would at least appreciate some indication of where I should start to modify it.

[YueYANG1996]

Same way as the microwave solution, adding another step to double-check the object selection plan here. You can implement a function taking in the original object plan and your prompt, and let a LLM to modify the original plan to satisfy your requirement.