How to change the parameters of a mujoco model on the fly?

[sarvghotra]

I asked the same question on Gitter channel but didn't get a solution. So I am putting it here.

By parameters I mean like the gear of a motor, length of a part of the agent, friction etc. Is there a way to change these on the fly through mujoco-py?

[tlbtlbtlb]

Changing model parameters is not supported, even by Mujoco's C API (which mujoco-py is a wrapper around). See http://mujoco.org/book/overview.html#Instance. You can change model parameters between simulation runs by creating new XML files and loading them. For example, you could modify the line https://github.com/openai/gym/blob/master/gym/envs/mujoco/half_cheetah.py#L7 to create and load a new XML file each time you create an environment.

[aravindr93]

@tlbtlbtlb Are you sure about this?

There are some model parameters in env.model.data for example. I am not sure if changing those actually affects the simulation or if it is just a dictionary for keeping track. But this is a very useful functionality for a number of applications. For example, you may want to test the robustness of the agent using perturbations on these model parameters. It would be a shame if we are required to generate multiple xml files to do this simple test. Thanks.

[tlbtlbtlb]

The supported mechanism is changing the model XML file, which is simple and fast.

If you change things inside env.model or env.model.data, those are internal data structures to Mujoco. There is a compilation step between the XML file and the data structure where a single number in the XML file might get translated into several matrices. It may be very hard to change some parameters correctly. So you really need read the Mujoco manual (start with http://mujoco.org/book/modeling.html#Compile) and ask their maintainers for advice about this.

[tlbtlbtlb]

(Don't actually change the XML file in place: write new files to /tmp and load them)

[aravindr93]

@tlbtlbtlb We asked this in the MuJoCo forum, and as we suspected, this functionality is actually available in MJC. We can modify any model parameter on the fly, and these changes take effect before a step is taken in the environment.

http://www.mujoco.org/forum/index.php?threads/can-the-parameters-of-a-model-be-changed-on-the-fly.3354/

The same when done through mujoco-py doesn't take effect -- I assume this is because the wrapper doesn't communicate back with the mjModel. Would it be possible to add this functionality, if it isn't too much trouble? My guess is that it shouldn't be hard.

[tlbtlbtlb]

Can you give an example of what doesn't take effect?
if you do, for example,

    x = model.body_mass
    x[1] = 3.6
    model.body_mass = x

does that work?

[tlbtlbtlb]

The array you get from reading model parameters is read-only, but you can make a copy of it and assign it back. This looks like it works:

>>> import gym,numpy as np
>>> e=gym.make('HalfCheetah-v1')
[2016-07-14 15:11:55,441] Making new env: HalfCheetah-v1
>>> mb = e.model.body_mass
>>> mb
array([[ 0.        ],
       [ 6.36031332],
       [ 1.53524804],
       [ 1.58093995],
       [ 1.0691906 ],
       [ 1.42558747],
       [ 1.17885117],
       [ 0.84986945]])
>>> mb = np.array(mb) # make mutable copy
>>> mb[1,0] = 7.7777
>>> mb
array([[ 0.        ],
       [ 7.7777    ],
       [ 1.53524804],
       [ 1.58093995],
       [ 1.0691906 ],
       [ 1.42558747],
       [ 1.17885117],
       [ 0.84986945]])
>>> e.model.body_mass = mb
>>> e.model.body_mass
array([[ 0.        ],
       [ 7.7777    ],
       [ 1.53524804],
       [ 1.58093995],
       [ 1.0691906 ],
       [ 1.42558747],
       [ 1.17885117],
       [ 0.84986945]])

[jietang]

@aravindr93 does this workaround work for you?

[machinaut]

model parameters are now read-write! (for better or worse)

Note that this doesn't guarantee MuJoCo detects or respects the update.

I recommend comparing dynamically changed models to re-loaded XML models, to ensure the behavior is sufficiently similar.

[xinleipan]

@machinaut Hi, could you explain more by read-write? does it mean you can change model parameters after you created the environment in python?

[machinaut]

Here's an example, where the gravity vector (a model parameter) is changed on-the-fly, resulting in a ball rolling around the floor: https://gist.github.com/machinaut/9bd1473c763554086c176d39062700b0

Read-write is just how we handle the accessing the memory which model parameters are stored. The arrays (e.g. in the given example sim.model.opt.gravity[:]) can be updated.

[vigneshramk]

So I tried emulating the things that were suggested above and this is what I notice:

1. The current version of mujoco-py (1.50) doesn't support read-write of model parameters like say body_mass, geom_friction etc. the model.opt parameters are the only ones that seems to be the only ones which is read-write.

2. However, the previous version of mujoco-py (0.5.7) with the downgraded gym version (0.7.4) is able to support read-write of parameters like body_mass, geom_friction etc. (like in the example given above).

Is there a way to make the model parameters (such as body_mass and so on) read-write with the latest version of mujoco-py?

Thanks for the help!

[joleeson]

My observations are the same as vigneshramk. @aravindr93 @sarvjeetsinghghotra would you care to share how you eventually handled the parameter changes for your study back in 2016? Was your implementation based on mujoco-py 0.5.7? Thank you in advance!

[sritee]

@algebraic-mouse I was able to modify the parameters in-place, like model.opt.gravity[-1] = 0

[SmileLab-technion]

i was able to modify all xml parameters(mass,geom,...) with mujoco-py 1.50 .

[HareshKarnan]

The array you get from reading model parameters is read-only, but you can make a copy of it and assign it back. This looks like it works:

>>> import gym,numpy as np
>>> e=gym.make('HalfCheetah-v1')
[2016-07-14 15:11:55,441] Making new env: HalfCheetah-v1
>>> mb = e.model.body_mass
>>> mb
array([[ 0.        ],
       [ 6.36031332],
       [ 1.53524804],
       [ 1.58093995],
       [ 1.0691906 ],
       [ 1.42558747],
       [ 1.17885117],
       [ 0.84986945]])
>>> mb = np.array(mb) # make mutable copy
>>> mb[1,0] = 7.7777
>>> mb
array([[ 0.        ],
       [ 7.7777    ],
       [ 1.53524804],
       [ 1.58093995],
       [ 1.0691906 ],
       [ 1.42558747],
       [ 1.17885117],
       [ 0.84986945]])
>>> e.model.body_mass = mb
>>> e.model.body_mass
array([[ 0.        ],
       [ 7.7777    ],
       [ 1.53524804],
       [ 1.58093995],
       [ 1.0691906 ],
       [ 1.42558747],
       [ 1.17885117],
       [ 0.84986945]])

was this done using mujoco-1.50 ?

[KitanoFumiya]

Hi!
I'm trying to change a length of parts of the agent on the fly by using Python, but I can't solve.
Does anyone know how to change the parameters?

[silverjoda]

You can change many of the parameters in the model on the fly and a lot of it works. I've used it using mujoco_py as well. You can expect some wierd behavior though if you change certain parameters on the fly during simulation. Some other things such as meshes and probably other object definitions can not be changed because they are compiled at runtime so changes have no effect.

[KitanoFumiya]

You can change many of the parameters in the model on the fly and a lot of it works. I've used it using mujoco_py as well. You can expect some wierd behavior though if you change certain parameters on the fly during simulation. Some other things such as meshes and probably other object definitions can not be changed because they are compiled at runtime so changes have no effect.

thank you for your comment.
Is there a short sample code if you like?

[Likangxidian]

The array you get from reading model parameters is read-only, but you can make a copy of it and assign it back. This looks like it works:

>>> import gym,numpy as np
>>> e=gym.make('HalfCheetah-v1')
[2016-07-14 15:11:55,441] Making new env: HalfCheetah-v1
>>> mb = e.model.body_mass
>>> mb
array([[ 0.        ],
       [ 6.36031332],
       [ 1.53524804],
       [ 1.58093995],
       [ 1.0691906 ],
       [ 1.42558747],
       [ 1.17885117],
       [ 0.84986945]])
>>> mb = np.array(mb) # make mutable copy
>>> mb[1,0] = 7.7777
>>> mb
array([[ 0.        ],
       [ 7.7777    ],
       [ 1.53524804],
       [ 1.58093995],
       [ 1.0691906 ],
       [ 1.42558747],
       [ 1.17885117],
       [ 0.84986945]])
>>> e.model.body_mass = mb
>>> e.model.body_mass
array([[ 0.        ],
       [ 7.7777    ],
       [ 1.53524804],
       [ 1.58093995],
       [ 1.0691906 ],
       [ 1.42558747],
       [ 1.17885117],
       [ 0.84986945]])

I have tried this way and it works. Thank you.

[yusukeurakami]

@Likangxidian
What version of Mujoco-py and Gym are you using??
I am using Mujoco-py 2.0.2.5 and Gym 0.15.7 but it returns "AttributeError: attribute 'body_mass' of 'mujoco_py.cymj.PyMjModel' objects is not writable".

[Likangxidian]

@Likangxidian
What version of Mujoco-py and Gym are you using??
I am using Mujoco-py 2.0.2.5 and Gym 0.15.7 but it returns "AttributeError: attribute 'body_mass' of 'mujoco_py.cymj.PyMjModel' objects is not writable".

@yusukeurakami
Hey, my gym version and mujoco-py version are 0.10.5 and 0.5.7, respectively. You may try it and it works in my computer.

[Likangxidian]

Hey, my gym version and mujoco-py version are 0.10.5 and 0.5.7, respectively. You may try it and it works in my computer.

…

------------------&nbsp;原始邮件&nbsp;------------------ 发件人:&nbsp;"Yusuke Urakami"<notifications@github.com&gt;; 发送时间:&nbsp;2020年3月21日(星期六) 中午1:15 收件人:&nbsp;"openai/gym"<gym@noreply.github.com&gt;; 抄送:&nbsp;"江山何处奏管弦"<xdlk@foxmail.com&gt;;"Mention"<mention@noreply.github.com&gt;; 主题:&nbsp;Re: [openai/gym] How to change the parameters of a mujoco model on the fly? (#234) @Likangxidian What version of Mujoco-py and Gym are you using?? I am using Mujoco-py 2.0.2.5 and Gym 0.15.7 but it returns "AttributeError: attribute 'body_mass' of 'mujoco_py.cymj.PyMjModel' objects is not writable". — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[a-z-e-r-i-l-a]

is it also possible to change friction related parameters such as impratio ? if I have noticed correctly, there was not method/attribute in sim.model for it.