By Pierre Nagorny – ARTANIM
Figure 1: a physics-based character (solid) walking next to the reference motion it was trained on (transparent). The physics controller drives the skinned character directly.
In a previous post, we asked how we move, and why a movement controller that matches our expectations is so hard to build. Since then, we at Artanim have focused on a more practical question: can physics-based characters be fast to produce and easy to share? We present our answer as a live demo at IEEE ISMAR 2026, and you can already try it at artanim.ch/artichar.
Most virtual characters replay recorded animations. A physics-based character is instead a simulated hierarchy of connected rigid bodies, moved at every instant by a small neural network- i.e. the controller – which has learned to reproduce a motion while keeping its balance. Throw a ball at it, and it stumbles and recovers. Yet these controllers are almost always shown on mannequins made of boxes and capsules, which hide many flaws in the movement. If we’re aiming to use such controllers to animate virtual humans in VR experiences, they need to be able to generate natural animation for realistic, skinned and textured characters.
Fast to train, easy to use: one controller for Unity and the web
Our pipeline takes a single animated FBX as input, builds its simulated body and trains the controller automatically. Training runs entirely on the GPU with JAX and MuJoCo MJX: 4,096 copies of the character train in parallel, in 30 minutes on a GPU. Each controller accumulates over 30 days of practice, with no manual tuning. To show the diversity of motion styles the method can reproduce, we trained 16 different walks on two characters of different sizes, from reference motions recorded with high-quality motion capture.
The same controller runs in Unity and on a web page with WebXR, with the same accuracy: hands and feet stay on average within about 1.5 centimeters of the reference mocap motion. In the browser of a standalone Meta Quest 3, the demo comfortably runs at a headset-native 72 frames per second with one character. The neural network controller is only a 1 MB download. No app store, no installation: a URL or a QR code is enough.
Figure 2: eight physics-based controllers, one per walking style, running side by side in the web demo. In VR, users can throw balls, add obstacles, and switch between motion styles.
What Comes Next: One Controller for Many Movements
Our ISMAR demo uses one controller per walking style. Next, in the PRESENCE SDK, we are building a single, general locomotion controller: a character that can adapt its movement to the people around it. It extends our demo with the University of Hamburg at ISMAR 2025, where a physics-based virtual agent kept its social distance and reacted when pushed. Can virtual humans share a space with us as naturally as another person would? This is what we are working toward in the EU Project PRESENCE. For more information, visit artanim.ch/project/presence.
ISMAR 2026 Demo: Physics-based controller policies driving VR characters
Joan Llobera, Pierre Nagorny and Caecilia Charbonnier – Artanim Foundation
Try the web demo: https://artanim.ch/artichar/ (in a web browser, or in the Meta Quest 3 web browser)
PRESENCE is co-funded by the European Union under Grant Agreement No. 101135025.
Tags: avatars, motion, physics-based movement, virtual humans, WebXR, ISMAR, VR, XR
