entity sampler repeating first object sample

[Kevin-Delnoije]

the entity sampler correctly outputs the image where model has correct rotations, however the rotations sampled during the first frame are repeated and each rendered image is exactly the same (0.hdf5, 1.hdf5 ......)

i have no idea why this is happening the config used for generating the data is here:

# Args: <cam_file> <obj_file> <output_dir>
{
  "version": 3,
  "setup": {
    "blender_install_path": "/home_local/<env:USER>/blender/",
    "pip": [
      "h5py"
    ]
  },
  "modules": [
    {
      "module": "main.Initializer",
      "config":{
        "global": {
          "output_dir": "<args:2>"
        }
      }
    },
    {
      "module": "loader.ObjectLoader",
      "config": {
        "path": "<args:1>"
      }
    },
    {
      "module": "lighting.LightLoader",
      "config": {
        "lights": [
          {
            "type": "POINT",
            "location": [5, -5, 5],
            "energy": 1000
          }
        ]
      }
    },
    {
      "module": "camera.CameraLoader",
      "config": {
        "path": "<args:0>",
        "file_format": "location rotation/value",
        "default_cam_param": {
          "fov": 1
        }
      }
    },
    {
          "module": "manipulators.EntityManipulator",
          "config": {
            "selector": {
              "provider": "getter.Entity",
              "conditions": {
                "name": "OfficeBranch_v5.002",
                "type": "MESH" # this guarantees that the object is a mesh, and not for example a camera
              }
            },
            "rotation_euler": {
              "provider": "sampler.Uniform3d",
              "max":[1.5, 0, 0],
              "min":[1.6, 0, 6.28]
        }
        }
    },
    {
      "module": "renderer.RgbRenderer",
      "config": {
        "output_key": "colors",
        "samples": 5,
        "render_normals": True,
        "normals_output_key": "normals",
        "render_depth": True,
        "depth_output_key": "depth"
      }
    },
    {
      "module": "writer.Hdf5Writer",
      "config": {
        "postprocessing_modules": {
          "depth": [
            {
              "module": "postprocessing.TrimRedundantChannels",
            }
          ]
        }
      }
    }
  ]
}

[themasterlink]

the entity sampler correctly outputs the image where model has correct rotations, however the rotations sampled during the first frame are repeated and each rendered image is exactly the same (0.hdf5, 1.hdf5 ......)

Thanks for providing the config file.

The entity sampler only samples one new rotation and does not create several rotations per key frame.

The suggested way of getting several different rotations is for now starting the BlenderProc process several times.

Does this answer your question?

[Kevin-Delnoije]

thanks, yes this answers my question!
although is there a way of inserting keyframes? so that you can render an animation, that seems like a more efficient way to get this done.

[themasterlink]

There is, if you already have the whole movement, you can set each frame with functions like these:

BlenderProc/src/object/ObjectPoseSampler.py

Lines 127 to 128 in 48ddfee

	obj.keyframe_insert(data_path='location', frame=frame_id)
	obj.keyframe_insert(data_path='rotation_euler', frame=frame_id)

Where frame_id is the number of the frame you want to use, keep in mind that blender starts at one, even though our outputs start at zero. You have to change the location and rotation_euler before calling this keyframe_insert function to save the position.

If you only have the first point and the end point, you can use blenders automatic interpolation, by only setting the first point and end point via the functions above.

Does this answer the question?

PS: We will add a module to change the position of objects based on files. Thanks for the idea.

[Kevin-Delnoije]

going to try this out!, I will let you know if this solves my problem

[Kevin-Delnoije]

on my desktop (ubuntu 20.04) i ran into a installation issue where skimage was not detected.
it was solved by changing this line

general_required_packages = ["pyyaml==5.1.2", "Sphinx==1.6.5", "scikit-image"]

for the object pose sampling i ran into the same problem that only one pose is sampled and this is expected since only one keyframe is inserted in object pose sampling?
if i set "cp_sample_pose": True, the pose of the model does not change.

so for my purpose of rotating the model i should create a copy of ObjectPoseSampler where i overwrite the insert_key_frames method?

# Args: <cam_file> <obj_file> <output_dir>
{
  "version": 3,
  "setup": {
    "blender_install_path": "/home_local/<env:USER>/blender/",
    "pip": [
      "h5py"
    ]
  },
  "modules": [
    {
      "module": "main.Initializer",
      "config":{
        "global": {
          "output_dir": "<args:2>"
        }
      }
    },
    {
      "module": "loader.ObjectLoader",
      "config": {
        "path": "<args:1>"
      }
    },
    {
      "module": "lighting.LightLoader",
      "config": {
        "lights": [
          {
            "type": "POINT",
            "location": [5, -5, 5],
            "energy": 1000
          }
        ]
      }
    },
    {
      "module": "camera.CameraLoader",
      "config": {
        "path": "<args:0>",
        "file_format": "location rotation/value",
        "default_cam_param": {
          "fov": 1
        }
      }
    },
    {
      "module": "object.ObjectPoseSampler",
      "config": {
        "max_iterations": 1000,
        "objects_to_sample": {
          "provider": "getter.Entity",
          "conditions": {
                "cp_sample_pose": True,
                "name": "OfficeBranch_v5.002",
                "type": "MESH" # this guarantees that the object is a mesh, and not for example a camera
          }
        },
        "pos_sampler":{
              "provider": "sampler.Uniform3d",
              "max": [5,5,5],
              "min": [-5,-5,-5]
        },
        "rot_sampler": {
          "provider": "sampler.Uniform3d",
          "max": [0,0,0],
          "min": [6.28,6.28,6.28]
        }
      }
    },     
    {
      "module": "renderer.RgbRenderer",
      "config": {
        "output_key": "colors",
        "samples": 5,
        "render_normals": True,
        "normals_output_key": "normals",
        "render_depth": True,
        "depth_output_key": "depth"
      }
    },
    {
      "module": "renderer.SegMapRenderer",
      "config": {
        "map_by": "instance"
      }
    },
    {
      "module": "writer.Hdf5Writer",
      "config": {
        "postprocessing_modules": {
          "depth": [
            {
              "module": "postprocessing.TrimRedundantChannels",
            }
          ]
        }
      }
    }
  ]
}

[themasterlink]

on my desktop (ubuntu 20.04) i ran into a installation issue where skimage was not detected.
it was solved by changing this line

Your solution works, but is not the desired one, you should have added it to the packages used in the setup:

 "setup": {
    "blender_install_path": "/home_local/<env:USER>/blender/",
    "pip": [
      "h5py",
      "scikit-image"
    ]
  },

Many examples do this already, check them for reference.

For your purpose, you need to create a new modul, which sets the location/rotation_euler for each obj and then saves the keyframe. You can not use the ObjectPoseSampler for that.

[Kevin-Delnoije]

Made a simple rotate module that is able to rotate the object, however still the first rotation is repeated even though i insert keyframes.

import bpy

from src.main.Module import Module
from src.utility.BlenderUtility import get_all_mesh_objects


class ObjectOriginRotater(Module):
    """
        Example 1: rotate model around origin

        {
          "module": "object.ObjectOriginRotater",
          "config":{
            "num_frames": 1000,
            "objects_to_sample": {
              "provider": "getter.Entity",
            },
            "rot_sampler": {
              "provider": "sampler.Uniform3d",
              "max": [0,0,0],
              "min": [6.28,6.28,6.28]
            }
          }
        }

    .. csv-table::
        :header: "Parameter", "Description"

        "objects_to_sample", "Here call an appropriate Provider (Getter) in order to select objects. Type: Provider. "
                             "Default: all mesh objects."
        "num_frames", "number of keyframes"
        "rot_sampler", "Here call an appropriate Provider (Sampler) in order to sample rotation (Euler angles 3d "
                       "vector) for each object. Type: Provider."
    """

    def __init__(self, config):
        Module.__init__(self, config)

    def run(self):
        num_frames = self.config.get_int("num_frames", 1000)
        objects = self.config.get_list("objects_to_sample", get_all_mesh_objects())

        # for every selected object
        for obj in objects:
            if obj.type == "MESH":
                print("Trying to put ", obj.name)

                for i in range(num_frames):
                    rotation = self.config.get_vector3d("rot_sampler")

                    obj.location = (0, 0, 0)
                    obj.rotation_euler = rotation
                    obj.keyframe_insert(data_path='location', frame=i+1)
                    obj.keyframe_insert(data_path='rotation_euler', frame=i+1)

    {
           "module": "object.ObjectOriginRotater",
          "config":{
            "num_frames": 1000,
            "objects_to_sample": {
              "provider": "getter.Entity",
            },
            "rot_sampler": {
              "provider": "sampler.Uniform3d",
              "max": [0,0,0],
              "min": [6.28,6.28,6.28]
            }
          }
        },

[themasterlink]

"max": [0,0,0],
"min": [6.28,6.28,6.28]

I am not sure what happens if min is bigger than max.

Did you print the rotations?

[Kevin-Delnoije]

oops, yes i printed rotations and they were different

<Vector (5.8349, 1.1485, 0.4405)>
<Vector (4.0852, 5.0130, 0.9347)>
<Vector (5.4031, 4.5572, 1.7349)>
<Vector (1.1056, 4.0624, 6.1266)>
....

[themasterlink]

So I tested your code and I only found a minor bug:

obj.keyframe_insert(data_path='location', frame=i)
obj.keyframe_insert(data_path='rotation_euler', frame=i)

For me it is working, I am not sure what your setup is, but copy pasting your code worked for me.

[themasterlink]

One addition:
Have you done it in "Debug" mode, check the examples/debugging for how to use BlenderProc directly in Blender, which shows you much better, which changes your code actually did.

[themasterlink]

@Kevin-Delnoije Have you been able to fix it?

[Kevin-Delnoije]

it is working now!
the debug tip is really helpfull