Repository navigation
MuJoCo 4: Studio & Filament, Deformables, Speculative Contacts #3773
Replies: 5 comments
|
insane release. congratulations! |
|
Wow, really great update! Very happy to have been following MuJoCo's development the last few months. Really cool to have the GUI supporting hot-reload. Thats actually a feature I built into my toolkit for building MuJoCo models (shameless plug) and found that it really improved the user experience to quickly work through building models. I've had some bugs getting the standalone studio working, but having a first-party web viewer is huge for remote development! If you're interested, I have been building my tool to provide a clean way to build complex MuJoCo models with Monte Carlo or optimization (mainly focused on 6-DOF rigid body mechanics), provide an easy way to output simulation data, and track progress while jobs run. MuJoCo is a super flexible platform to build off of, so hats off to the team, and many thanks to all the contributors for building MuJoCo! |
|
this is such a great update |
|
Amazing! Thank you so much for the best sim in the world! |
|
so excited... scheming |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Three years ago, MuJoCo 3.0 introduced MJX, signed distance fields, constraint islands and early deformables. There have been 39 releases since then. MuJoCo now gets about 3.6 million PyPI downloads a month, has more than 16K GitHub stars, and in the past year more than 45 external contributors submitted around 2K commits. We have also been working with NVIDIA on MuJoCo Warp, and with NVIDIA, Disney, TRI and others on Newton (under the Linux Foundation).
Since
3.5, the middle version number simply marks breaking changes, so we saved4.0for a big step. Today we are releasing MuJoCo 4, together with MuJoCo Playground 1.0.Getting Started
pip uninstall -y mujoco # if upgrading from 3.x pip install -U mujocoTo see the new features without installing anything, visit mujoco.org, which is now an interactive 4.0 showcase running in your browser, or open any model at live.mujoco.org. The Python package is now split into a minimal
mujoco-coreand optional components, andmujocoinstalls all of them. Samples andmodel/scenes are omitted from the PyPI wheels to keep them small—expand the block below to run the same feature slideshow locally from a repo checkout.Install options & feature slideshow: minimal headless installs, extras, robot models and showcase
Then and Now
If you last looked closely at MuJoCo around 3.0, this is what changed. The changelog has the per-release details.
simulate/mujoco.viewer, OpenGL 1.5flex, small timesteps, elastic onlymarginlibccdmujoco.batchmulti-world CPU, MJX, MuJoCo Warp (with NVIDIA), browser (WebAssembly,@mujoco/mujocoon npm)motor,position,velocity,muscle, …pid, datasheetdcmotor,SO(3)orientation, delays,tactile,contact, camerarangefinderMjSpec,<frame>,<replicate>,<attach>,.mjzarchives, non-destructive saving, Studio hot-reload for live human & agent authoringpip install mujoco-menageriemujoco.sysid, MuJoCo Playground 1.0mujoco-core(numpy only) plus optional components, Python 3.11–3.15 including free-threaded buildsPillar 1: MuJoCo Studio (
mujoco.studio), Filament PBR &live.mujoco.orgFor a decade, MuJoCo shipped a single fixed-function OpenGL 1.5 viewer (
simulate/mjr, now in maintenance mode), and extending a monolithic viewer proved a poor fit for custom research tools. In 4.0, MuJoCo Studio graduates out ofexperimental/with a modular, viewer-agnostic MuJoCo Studio SDK for Python and C++ at its core:hot_reloadlive-swaps PythonViewerPlugin/SimPlugincode and MJCF files. Paired with shipped coding-agent skills, these lightweight, composable APIs make it easy to spin up custom visualizers and debug tools per experiment.live.mujoco.org): Built on top of the SDK, our new default interactive viewer (mujoco::studio::Appin C++,studio_app.Appin Python, and in the browser at live.mujoco.org with.xml,.mjb, and.mjzdrag-and-drop plus?model=github:<org>/<repo>/<branch>/<path>deep links) adds dockable light/dark workspaces, an integrated console, a 3D camera gizmo, andMjSpecinspection and editing. The Studio App is optional: in Python,studio_app.App()is just an ordinary plugin with no special status, so you can use it out of the box, extend it with your own plugins, or leave it out altogether to build a custom tool, like the feature slideshow that now powers mujoco.org.--gfx=web): Run Studio on a remote or GPU-less machine, open it in a browser, and share the session with colleagues. One browser at a time is the controller and drives the simulation and UI; everyone else watches the same session live as a spectator, and control can be handed over (WebViewer). The same tools and plugins also run as native desktop apps (NativeViewer), with no change to simulation or plugin code.mjrf): Powers 3D rendering in the SDK and standalone offscreen views with metallic-roughness materials, HDR image-based lighting (<light type="image" texture="..."/>), soft area shadows, tone mapping/AO/bloom/fog post-processing, and asynchronous multi-camera RGB/depth/segmentation frame queues.Code sample: Launch Studio from the command line or Python
# Native desktop window or headless remote browser viewer (with live plugin & XML hot-reload) python -m mujoco.studio.viewer --model=humanoid.xml --hot_reload=all python -m mujoco.studio.viewer --model=humanoid.xml --gfx=webTry it: a humanoid in your browser.
Pillar 2: Deformables Have Matured
Soft bodies in MuJoCo 4.0 graduate from experimental plugins and explicit penalties to native continuum deformables solved implicitly alongside rigid bodies, joints, and tendons:
discreteintegrator: Folds passive damping and elasticity into the constraint solver's effective inertia (M_hat = M + h*D + h^2*K), keeping stiff servos, cables, thin shells and 3D solids stable at 5–10x larger timesteps (experimental large-strain Neo-Hookean solids are also available viaMjSpec). Because cloth stretching is now integrated implicitly, the oldflexvertequality workarounds are retired.bag_hoop.xml,poncho.xml).springyield(wire.xml). 2D shells and 3D solids, includingtrilinearandquadraticcages, yield in stretching (von MisesJ2) via<elasticity yield>(clay.xml,roller.xml,torsion.xml). Shells withelastic2d="bend"or"both"also yield in bending, so folded paper and metal foil keep their creases and dents (paper.xml,foil.xml). Plastic state lives inmjData.plasticand is saved withmjSTATE_PLASTIC.dim="1"flexes are discrete elastic rods with bending and twist (section="circle|box",shear, curved rest shapes), matching Euler-Bernoulli cantilever deflection within 0.5% on both CPU and MuJoCo Warp (cable.xml,coil.xml,belt.xml). They replace<composite>and the cable plugin.trilinearandquadraticcages integrate stiffness and mass over the true mesh volume (Finite Cell Method), distribute contact forces with exact barycentric weights, take<mesh>assets directly (<flexcomp type="mesh" mesh="...">), and support per-element materials.Several of these are a single attribute on an existing model:
wire.xml)clay.xml)paper.xml)foil.xml)Try it: clay, wire, paper, foil, drawstring bag, cable.
Pillar 3: Rebuilt Collisions & Speculative Contacts (
<flag speculative="enable"/>)Across the 3.x releases we rebuilt the collision stack:
nativeccdreplacedlibccd, and single-shot multicontact (multiccd) has expanded to all geom types (including withmargin), resulting in a ~4x speedup and significant robustness improvements over multicontact withlibccd. The entire collision stack (geoms, SDFs, and flexes) is now fully multithreaded, large-mesh convex collisions got 2x faster, the box-box collider was rewritten, and new contact features include per-contact surface velocity for conveyors andadhesionfor magnets and suction. Right: box meshes in single precision withlibccd(left) andnativeccd(right).In 4.0 this culminates in speculative contacts, which prevent thin or fast-moving objects from tunneling through other objects. Speculative contacts sweep fast-moving geoms ahead of time (costing almost nothing for slow ones), keep bodies on the separated side, and generate up to four contact points so thin cards and plates land flat, all without sub-stepping.
In our tests, a robot arm catches a 0.7 mm playing card falling at over 2 m/s every time with a 16 ms timestep, where discrete collision detection misses it at anything above 8 ms. Humanoid stair climbing and thin-object grasping run 6 to 9 times faster, because they can use larger timesteps. In 4.0, speculative contacts cover convex primitives and meshes on CPU; heightfields, SDFs, flexes, MJX and MuJoCo Warp will follow.
Click a title to run it in your browser; turn off
speculativeto watch the objects pass through each other.Upgrading from 3.x
Most models load unchanged. The changes most likely to affect you are below; the 4.0.0 release notes have the full list and migration recipes.
pip uninstall -y mujocobefore installing 4.0 (see Getting Started).mujoco.experimental.studiois nowmujoco.studio; the default viewer plugin ismujoco.studio.app.App.<composite>, themujoco.elasticitycable and shell plugins, andflexvertequalities are removed in favor of<flexcomp>and continuum<elasticity>. 1D rods needintegrator="discrete",EulerorRK4.Thank You to Our Community
MuJoCo 4.0 would not exist without the bugs reported, designs debated, and code contributed by our open-source community and ecosystem partners—including our collaborators at NVIDIA and Disney Research on
mujoco_warpand Newton, and everyone who contributed acrossmujoco,mujoco_warp, andmujoco_menageriesince3.0:External contributors across the 3.x–4.0 cycle (click to expand)
@adenzler-nvidia,@aftersomemath,@alberthli,@Alex108306,@alper111,@AndreasKuhner,@Andrew-Luo1,@ayonga,@chenxin199305,@chernyadev,@christianrauch,@ctkuan-flexiv,@dongridong,@eholum-nasa,@eufrizz,@gizemozd,@hello-chintan,@hello-fazil,@hshi74,@johnnynunez,@jonathanembleyriches,@jonzamora,@juelg,@Kallinteris-Andreas,@kevinzakka,@kimtaehyeong99,@lonelyfluency,@lorenzo96-cmd,@louislelay,@lvjonok,@M-Colley,@mar-yan24,@michael-ahn,@mmporong,@muhrix,@n3b,@Nan-Tetheria,@nvtw,@omarrayyann,@qiayuanl,@rishabhsai,@s1lent4gnt,@saikishor,@spjardim,@sylvesterkaczmarek,@teerthsharma,@thomaspeyrucain,@tlpss,@toekneema,@tomstewart89,@v-r-a,@varadVaidya,@yanansui,@yinfanyi,@zchJoTell us what you are building, share scenes via live.mujoco.org, and ask questions in the thread below!
— The Google DeepMind Robotics Simulation Team
All reactions