CI Configuration Model Definitions
Let's desect a model definition file step by step
[TOC]
Then let's have a look the rest of the model properties: Those define the name the modelname and type. Also you can here define MR properties like seen in the pipeline.yml.
...
model: # now we are talking about a model that has more than one parent.
robotname: mighty_bust
modelname: example_model
...Each model is derived from one or more base models.
...
input_models: # the input models with pathes given relative to the pipeline-definition file
torso:
basefile: "../torso-base-urdf/urdf/model.urdf"
# Here comes a second parent model, but this is not a base model its one that was already derived.
right_arm:
derived_base: "right_arm_model.yml" # has to be processed before
head:
repo: # you can also derive from a git model
git: ...
commit: ...
urdf_in_repo: ...
...NOTE When using an derived model as input. Make sure that this model is included in the [pipeline.yml] (Configuration/pipeline-definition)
model_definitionslist BEFORE this model definition.
CAUTION For derived models only the structure in the URDF is joined. The smurf definitions might be applied as well, but when names changed during the combination these might not be correct. It is recommended to set
inherit_smurf_properties: False
Now we have a new section: assemble Here we define how the input models will be joined together.
...
assemble: # here we assemble the model based on the inputs, all annotations present in the inputs will be inherited
model: torso
# remove_beyond: link_name and take_leaf: link_name can be used to reduce each model
children:
- model: right_arm
joint:
parent: right_arm_mount_link
type: fixed
- model: right_arm
joint:
parent: left_arm_mount_link
type: fixed
mirror:
flip_axis: 1
name_editing: # will take care of all referenced name occurrences in the join section
prefix: "a-prefix-to-prepend-to-everything"
collision_suffix: "a-prefix-to-append-to-all_collisions"
# prefixes and suffixes for link, joint, visual work accordingly
replacements: # Regular Expressions are possible
- "right_": "left_"
...Note The connection joints are defined by default as fixed with a zero-transformation.
For each model in the assemble section we can also define name_editing, take_leaf, remove_beyond and/or mirror keys explained in the following:
With the name_editing you can edit the names in your model.
...
name_editing:
replacements:
- "blablub": Text
collision_replacements: [] # like replacements
collision_prefix: PREFIX
collision_suffix: SUFFIX
visual_replacements: []
visual_prefix: PREFIX
visual_suffix: SUFFIX
link_replacements: []
link_prefix: PREFIX
link_suffix: SUFFIX
joint_replacements: []
joint_prefix: PREFIX
joint_suffix: SUFFIX
submechanism_replacements: []
submechanism_prefix: PREFIX
submechanism_suffix: SUFFIX
...If you want that the derived model is only a part of the input model. You can use the take-leaf parameter to continue the work only with the given part of the model.
For the entry you give only this link where to cut the model. When this is set the model includes only the given link and everything attached to this link further down the model tree.
...
take_leaf: $LINK_NAME # the link where to cut the model
...This option is similar to the take_leaf option but the more rootish version of of it.
Whith remove_beyond you can remove everything from the model that follows the given links.
You can either give a single link:
...
remove_beyond: $LINK_NAME # the link where to cut the model
...or a list of links:
...
remove_beyond: [$LINK_NAME1, $LINK_NAME2] # the link where to cut the model
...Note
remove_beyondis executed beforetake_leaf. Give them only if you want to use them.
To mirror a base model at the Y-plane simply add mirror: {}. If you want to mirror at another plane or define meshes that are symmetric and do not need to be mirrored define, it like this:
mirror:
plane: [0,1,0] # this is the default
exclude_meshes:
- "$MESH_NAME" # or ALL
flip_axis: 1 # One axis has to be flipped after mirroring to maintain right handed frames
name_replacements:
"right": "left"Note If you mirror a submodel, don't forget to apply the name replacements to the child_link in the definition
Under the export_config key you can define what exports you want to save from the created model.
A SMURF file which bundles the XML-model file and the annotations is always exported.
A SDF/URDF export must be configured at least once with link_in_smurf set to true.
...
export_config: # smurf will be exported by default, this list must not be empty
- type: urdf # sdf
mesh_format: stl # obj, mars_obj, bobj
additional_meshes: bobj # optional
link_in_smurf: True
include_cut_joints: False # possible for all but only valid for sdf types
filename_suffix: "" # possibility to append a suffix to ambiguate different urdf versions e.g.
ros_pathes: False
enforce_zero: False # sets values that are basically zero to zero
- type: pdf
- type: joint_limits
joints: ALL # "INDEPENDENT", "ACTIVE", submodel name
file_name: "joint_limits.yml"
- type: submodel # will export a submodel to submodels directory
add_floating_base: False # adds a floating base joint set to the root of the robot
start: "body_root_link" # if not given the root of the robot
stop: [ "right_arm_elbow_link", "left_arm_elbow_link" ] # if not given the leaves pf the robot tree
name: "torso_to_elbows"
export_config: # if not given, the same export_config is taken as for this model (submodels excluded)
- type: pdf
- type: joint_limits
joints: ALL # "ABSTRACT", "INDEPENDENT", "ACTIVE", "ACTUATED"
file_name: "joint_limits.yml"
- type: kccd # this defines a kccd model
join_before_convexhull: True
remove_joints:
# for the kccd model we have to reduce joints to speed up computation
- right_arm_mount_link
merge_additionally:
# here we can define where we want to merge collisions without removing the joint link
- ALShoulder2
keep_stls: False
keep_urdf: False
reduce_meshes: 0.5
braking_model:
linearFactor: 0
invDeceleration: 0.02
uncertainty: 0
latency: 0
simplify_swept_from: [ ALWristYaw_Link, ARWristYaw_Link ]
safety_distance: 0
report_up_to: 0
computation_budget: -1
max_approximation_order: 1
...If you want to export a second version of your model which provides all the filename-attributes in your URDF/SDF with the ROS style file path set the ros_pathes option true. If you also export a floating base model, then there will be generated a second version for this URDF, too
If you want to export a second version of your model with a floating base attached to the root of your model, then simply add add_floatingbase: True to the new export entry.
Submodels will be exported according to the main export_config if not explicitly specified agein in the entry.
With this, you can export ROCK-compatible joint_limit files. You can export it in several ways.
With the entry given by joints you can specify for which joints the limits are included in that file. Possible entries are a list of joint names, "ALL", "ACTIVE", and "INDEPENDENT". The latter two only work if submechanims are defined.
In the test job of the CI-pipeline the model will be tested. These tests are configured in the following:
...
test:
tolerances:
default: 0.01
mass: 0.01
rad: 0.03
distance: 0.003
compare_model:
# We also need the information with which model this export should be compared.
# Probably it's just the definition of the target repo.
git: "git@gitlab.com:models/robot-subgroup/output_repo"
branch: master
model_in_repo: "smurf/robot-model.smurf" # this is the path to the urdf in the target repo
# submechanisms_in_repo: "smurf/robot-model_submechanisms.yml"
ignore_failing_tests_for: "82b92a86" # a commit hash of the compare model where the faiiures will be ignored
tests:
- process_double_check
- load_in_pybullet
- compare_link_masses
- compare_link_transformations
- topological_self_consistency
- compare_amount_joints
- hyrodynChecks:
- load_in_hyrodyn
- compare_masses
- compare_com
- compare_torques
...CAUTION You can add the
ignore_failing_tests_fortag with the last commit hash of the target repo to overwrite it with the new model. The test will then be run, too, but if they fail the model will be pushed anyways to the master. The quotation marks are needed.
Now we define, how we derive the the model. We can do the actions listed here to derive the model.
The model_tests section defines the test procedure. The given tests are executed in that specific order. This is what each test does:
-
topological_self_consistency: This test will be run always! No matter if you listed it or not Checks if the new model is consistent. Are there enough joints for the number of links? And are there breaking changes between the latest version on master and the newly exported version? This includes removed, renamed links or joints or changed joint types. -
process_double_check: Checks if the defined operations from the model definitions have been performed -
load_in_pybullet: Tries to load this model in pybullet (e.g. will fail if not all meshes are present) -
compare_link_masses: Compares the masses of the links of the old model with those of the new model -
compare_amount_joints: Checks if the new and the old model have the same amount of joints.
hyrodynChecks:
These checks are only possible for models for which submechanism are defined in the mode definition.
-
load_in_hyrodyn: Tests if the model can be loaded in hyrodyn -
compare_masses: Compares the total mass of the old and the new model -
compare_com: Compares the COM position of the old and the new model -
compare_torques: For the current pose compares the joint torques of old and new model required to maintain this pose under gravity -
compare_link_positions: Compares the positions of the given links between old and new model. Expects a list of link names. -
symmetry_check: Checks if the given links in the new model are symmetric. Expects a list of two link names. -
move_hyrodyn_model: Moves all joints of both old and new model to the given joint angle. Expects either a float (applies to all joints, or a list of floats, for each joint one. Afterwards tests can be repeated. To perfrom those checks in the new pose.
For the deployment of the created and tested model we give the properties under the deployment key:
...
deployment:
do_not_deploy: False
mr_target_branch: develop # can override the one defined in pipeline.yml
mr_title: "New pipeline model" # can override the one defined in pipeline.yml
mr_mention: "@hwiedemann" # can override the one defined in pipeline.yml
keep_files: []
submodules:
- repo: ...
target: ...
commit: null
branch: master
mirror: # see Deploy to mirror section
submodules: False
repo: "git@gitlab.com:models/a_mirror_repo_for_this_model.git"
branch: "develop"
merge_request: False
...By default all files (except .git-files, scripts-directory, README.md and the manifest.xml) in the target repo are removed and then replaced by the newly generated ones. You can make files persistent by listing them under keep_files:
...
keep_files:
- "submechanisms/lambda-mechanism_knee.lua"
...NOTE These files will overwrite changes by the pipeline. Therefore do this only for files the pipeline does not generate.
By default all files including submodules in the target repo are removed in order to replace them with the current version of the defined ones. Therefore, submodules that shall persist have to be defined under submodules:
...
submodules:
- target: "a-submodule"
repo: "../a-submodule.git"
branch: master
...In case you want to mirror a result model to a second repository - maybe in the case that it is needed in another namespace - use the mirror key in the deployment configuration.
The submodules parameter gives you control over the submodules in the mirror repo.
If it is set to True the submodules will be set like in the normal target repo, too. When your other namespace doesn't has permission to access these submodules, set the parameter to False. This way the files will be included directly in the repo.
NOTE Due to an GitLab issue it is currently not possible to create a merge request automatically in forked repos. (It will open the MR not in the fork internally but tries to merge to the original repo). Therefore, when you are adressing a fork here, make sure to set
merge_request: Falseandbranch: developso that you can check and merge manually.
The annotations/actions that can be performed to specify changes in the derived model are listed below.
Note Those, which are enumerated are processed in the given order. that means that for the next action the changes of the one before apply. The unnumbered actions are performed afterwards but do not influence each other
- Frames
- Joints
- Collisions
- Sensors
- Poses
- Materials
- Submechanisms
- Further Annotations
- Running custom scripts
This entry gives you the ability to transform a link by the given transformation. This way you can reorient links that e.g have not been mirrored to your wishes or if you want to adapt. You can either transform the whole link including it's collision visuals and so on, or only the frame.
...
frames:
# For each link/frame we can here add annotations and perform transformations
$default: # these annotations/actions will be set to all links if not defined otherwise there
noDataPackage: False
reducedDataPackage: True
estimate_missing_coms: True # if the COM is not set this entry will do so
right_arm_mount_link:
transform_link:
# transform the link including its collisions, visuals and intertials by the given transformation
rpy: [ 0, 0, -1.5706463 ]
xyz: [ 1, 0, 0 ]
left_arm_mount_link:
transform_frame:
# transform only the frame of the link by the given transformation. Visuals, collisions and inertials
# stay where they are
rpy: [ 0, 0, -1.5706463 ]
xyz: [ 0, 0, 0 ]
control_frame: # the link you want to move
reparent_to: left_arm_mount_link # the link you want to attach it to
new_frame:
parent: root # the parent link to which the new link shall be added
jointname: new_frame_joint # optional, if not given the name is identitcal with the link
xyz: [0, 0, 0] # the translation from parent to the new link
rpy: [0, 0, 0] # the rotation from parent to the new link
mass: 1.0 # option to give this link a mass. inertia tensor is diag(1e-6)
...According to the URDF convention the inertials should have the origin of the links COM. Sometimes the base URDFs define the link COMs at the link's joint position. As this is not correct for the most cases you can set the estimate_missing_coms option to True.
Given this, the pipeline will estimate the COM positions for all links where the COM is coingurent with the joint based on the link's collision geometry.
See also https://wiki.ros.org/urdf/XML/link
Sometimes - especially for links that represent fixed frames - you want to move these links to another parent without changing there transformation relative to root. For this case use reparent_to:
NOTE If you reparent a link that is moveable, this will not yet be considered with this option. It will simply be transfered to maintain the root->Link transformation for the zero configuration of the robot.
Adding a frame means adding a fixed joint and a link. Exepects a list of links and joints to add. Simply by defining a frame that is not yet in the robot, it will be added.
Similar to frames you can edit all joints here. Joints not listed here remain unchanged. Here you can provide a dict of joints for which you want to override the values. You can also define mimic joints or define joints to be part of a hyrodyn transmission (see Hyrodyn Doc: http://bob.dfki.uni-bremen.de/apis/skumar/hyrodyn/classTRANSMISSION_1_1transmission.html), if they depend on multiple other joints.
...
joints:
$default:
# here you can give default values that'll be written to undefined values or
# for the limits to those which are zero
min: ["rad", -1]
max: ["deg", 180.0]
eff: 100.0
vel: 2*pi
$replace_joint_types:
prismatic: fixed
# Each joint can have an entry defining the joint parameters for changing the
# transformation of joints, see links section. Besides annotations like min, max,
# vel, eff you can define the joint as active: False or active: **motor_annotations
# with **motor_annotations being the dict of necessary information on that motor also
# you can define here joints that are cut_joints, for those you have to define parent
# and child and set cut_joint: True the transformation for them will be calculated automatically
body_joint_1:
min: -3.1415926
max: ["deg", 180.0]
eff: 400.0
vel: 2*pi
# any additional annotations are possible, too
noDataPackage: False
reducedDataPackage: True
# these entries override the existing values
active:
name: body_joint_1_motor
# from here on these are additional and optional annotations to this motor
p: 20
i: 0.001
d: 0.1
body_joint_2:
remove: True # removes this joint the rest here will be ignored
body_loop_closure_joint: # this joint doesn't exist in the input models
cut_joint: True
parent: body_link_1
child: body_link_2
active: False
left_arm_shoulder_joint1:
move_joint_axis_to_intersection: # Moves the joints to the intersection of the two given joints.
- left_arm_shoulder_joint1
- left_arm_shoulder_joint2
movement_depends_on: #alternati you can provide a single dependency as dict to the mimic key
- joint_name: left_exo_shoulder_1_joint
multiplier: -1
offset: 0.0
active:
name: motor1
p: 20.0
i: 0.0
d: 0.1
noDataPackage: False
reducedDataPackage: False
...Via the $replace_joint_types property you can replace a certain joint type by another. Useful when you have adjustable lengths in your CAD export that are fixed.
You can define the motor controller parameters for MARS using the following entry. The given values are the default values. You can overwrite the default values for specific joints by defining them again under the joint name.
Here we can apply some additional values that'll be set in the SMURF annotations.
When auto_bitmask is set to true all collision bitmasks are set automatically in such way that they there are no internal collisions in the robot. It overrides all bitmask definitions for the links. Also, for the DataPackage Management, the default will be taken.
Auto-bitmasking prevents collisions between:
- Collisions in the same link
- Collisions with the Collision in the parent frame
- Collisions with collisions in the parent's parent and above as long as the transformation between those links is only rotational
- Collisions where the bounding_boxes touch
...
collisions:
auto_bitmask: True # will generate a matrix which collisions are able to collide and set a bitmask for the collisions
no_collision_between: # here you can declare which collisions are unable to collide
body_joint_1_collision: ["right_arm_mount_link_collision"]
$default: # Any annotations that shall be added to the collisions by default
noDataPackage: True
shape: convex
left_arm_mount_link_collision:
remove: True
right_arm_link_collision:
shape: box
ccfm: 0.002
reducedDataPackage: True
bitmask: 1
...A dict of joint configuraionts to be saved in the smurf. Will only be applied when the model is of type smurf.
...
poses:
init: # name of the configuration
LLHip3 : -0.35
LLKnee : 0.64
LLAnklePitch : -0.28
LRHip3 : -0.35
LRKnee : 0.62
LRAnklePitch : -0.27
ALShoulder1 : -0.49
ALShoulder2 : 0.49
ALElbow : -0.30
ARShoulder1 : 0.49
ARShoulder2 : -0.49
ARElbow : 0.30
BodyPitch : 0.20
...Adding and editing materials is possible under the materials key:
...
materials:
# A dictionary of materials. entries that do not relate to an existing one are created and can be used as material for links
shiny_blue_material:
diffuse: [0.0, 0.0, 1.0]
ambient: [0.0, 0.0, 1.0]
specular: [0.0, 0.0, 1.0]
emissive: [0.0, 0.0, 1.0]
# texture: path/to/texture # [TODO pre_v2.1.0] the texture has to be copied (c.f. meshes)
...A dict of jsensor definitions to be saved in the smurf. Will only be applied when the model is of type smurf.
Possible sensors:
- Single Link sensors:
- Joint6DOF
- CameraSensor
- RotatingRaySensor
- IMU
- Multi Link sensors:
- JointPosition
- JointVelocity
- MotorCurrent
- NodePosition
- NodeRotation
- NodeContactForce
- NodeCOM
...
sensors: # or smurf_sensors
# Single Link sensors
Joint6DOF: # the sensor type
name: LLAnkle_ft # name of the sensor
link: LLAnkle_FT # the link to which this sensor belong
# Multi Link sensors
JointPosition: # the sensor type
name: Joint_Position # name of the sensor
targets: All # either a list of joints or "All" when it shall be applied t all moveable joints
...In the following is shown an example on how to define the submechanisms.
Therefore you have two possibilities: Either by the by the submechanisms_file key or directly by the submechanisms and exoskeletons keys.
The first way will let the pipeline manage the submechanisms and the fixed joints automatically.
Using submechanisms_file will directly output it's content to the final robot model's submechanisms file.
...
submechanisms_file:
submechanisms:
- name: "rh5_torso"
type: "2SPU+1U"
contextual_name: "torso_joint"
file_path: "Body.urdf"
jointnames_independent: ["BodyPitch", "BodyRoll"]
jointnames_spanningtree: ["BodyPitch", "BodyRoll", "Body_B11", "Body_B12", "Body_Act1", "Body_B21", "Body_B22", "Body_Act2"]
jointnames_active: ["Body_Act1", "Body_Act2"]
...
exoskeletons: # other definitions
...
...or
...
submechanisms:
- name: "rh5_torso"
type: "2SPU+1U"
contextual_name: "torso_joint"
file_path: "Body.urdf"
jointnames_independent: ["BodyPitch", "BodyRoll"]
jointnames_spanningtree: ["BodyPitch", "BodyRoll", "Body_B11", "Body_B12", "Body_Act1", "Body_B21", "Body_B22", "Body_Act2"]
jointnames_active: ["Body_Act1", "Body_Act2"]
...
exoskeletons: # exoskeletons definitions
...
...See also https://git.hb.dfki.de/dfki-mechanics/hyrodyn/hyrodyn/-/wikis/home
The pipeline offers you the possibility to run custom shell commands. Therefore, you have to add the following option. Those commands will be run after the new model has been exported, and before the old model is replaced. The commands will be executed in the given order. Make sure you have " and ' correct.
...
post_processing:
- cmd: 'echo "Hello World!"'
- cmd: "check_urdf urdf/model.urdf"
cwd: "temp/temp_full-urdf" # ***CAUTION*** default value, probably you don't want to change this
- cmd: "python ../../full-urdf/scripts/post_process_new_model.py urdf/new_model.urdf" # see below for explanation
...The last example shows the way you probably want to use this feature: You have the possibilty to store model related scripts in their repos. Call the python script in the target repository to make changes to the newly exported urdf. Remember the current working directory is the directory of the newly processed model.
See also the directory structure here
WARNING The commands are executed by default in the model temp directory which contains the newly processed model. This should be the only directory where you act and make changes! It is possible to change the working directory and to make changes elsewhere, but this will probably lead to unwanted behavior!
IMPORTANT You should not edit the meshes. Or you might affect other models
Back to top.
