Temporal Residual Jacobians for Rig-free Motion Transfer

[ECCV 2024]

1University College London 2Adobe Research 3University of Montreal


We introduce Temporal Residual Jacobians as a novel representation to enable data-driven motion transfer. Our approach does not assume access to any rigging or intermediate shape keyframes, produces geometrically and temporally consistent motions, and can be used to transfer long motion sequences. Central to our approach are two dedicated neural networks that individually predict the local geometric and temporal changes that are subsequently integrated, spatially and temporally, to produce the final animated meshes. The two networks are jointly trained, complement each other in producing spatial and temporal signals, and are supervised directly with 3D positional information. During inference, in the absence of keyframes, our method essentially solves a motion extrapolation problem. We test our setup on diverse meshes (synthetic and scanned shapes) to demonstrate its effectiveness in generating realistic and natural-looking animations on unseen body shapes.

Motion sequences

We showcase our method on a wide variety of sequences from AMASS and humanoid meshes with significantly different shape and topology. All results show transferred motion to an unseen shape without rigging.

Running (in-place)

Number of frames: 278


holey man



Number of frames: 187


holey man


Jumping jacks

Number of frames: 460

Notice secondary motion on the chubby man around the chest and belly regions.

chubby man




Number of frames: 1845

holey man


skeleton zombie


Number of frames: 1088

original capture (unsynchronized)

skeleton zombie


Generalization beyond humanoids

Temporal Residual Jacobians generalizes to scans (FAUST) and in the wild meshes (Mixamo).

FAUST scan (running)

FAUST scan (punching)

Mixamo skeleton zombie (running)

Mixamo skeleton zombie (punching)

Motion transfer from human to animal

Temporal Residual Jacobians can generalize motion transfer to animals even when it's trained only on humans.


Target (reptile)

Motion transfer on animals


DeformingThings4D is a synthetic dataset of animations including animals.

Source (bear)

Target (triceraptor)

Target (wolf)

Source (dog)

Target (triceratop)

Target (wolf)

COP3D: Common Pets in 3D

COP3D dataset contains in the wild animal reconstructions from consumer devices.

Original capture

Source (dog)

Target (wolf)

Target (reptile)

Target (triceratop)



In comparison to ours, NJF has jitters and artefacts especially at later time steps.

Vertex ODE (holey man)

NJF (holey man)

Ours (holey man)

Vertex ODE (scan)

NJF (scan)

Ours (scan)


We do not impose physics constraints, therefore, our animation can have self-intersections. An interesting direction is to incorporate constraints for collision detection.