USDCraft — Articulated 3D Assets for Simulation

Demo

Method

Generation

New articulated assets from text or images

Without any input mesh, USDCraft writes an asset program from a prompt or a reference image. Pretrained LLMs bring broad knowledge of everyday objects, so new categories need no task-specific training. Every result below is the compiled, articulated USD asset — drag to orbit; parts move through their joint limits.

Upright piano

Image → asset

USDCraftLoading 3D

Backyard playground set

Image → asset

USDCraftLoading 3D

Manual wheelchair

Text → asset

Prompt

Create a complete, functionally usable, simulator-ready manual wheelchair with two rotating drive wheels, swiveling front casters, fold-up footrests, push rims, brakes, seat, backrest, and armrests.

USDCraftLoading 3D

Parcel locker bank

Text + image → asset

Prompt

Create a complete, functionally usable, simulator-ready parcel locker bank with eight differently sized independently hinged doors, handles or electronic latches, kiosk screen, stable enclosure, and clear retrieval access.

USDCraftLoading 3D

Deployable satellite antenna

Text → asset

Prompt

Create a complete, functionally articulated, simulator-ready deployable satellite high-gain antenna assembly with a spacecraft mounting pedestal, two-axis azimuth-and-elevation gimbal, a parabolic reflector divided into a rigid center dish and two foldable side petals, locking hinge housings, central feed horn, support struts, cable-routing brackets, hard stops, and launch-restraint latches. The antenna must fold into a compact stowed configuration and deploy into a coherent dish shape, while the deployed reflector can aim through both gimbal axes without colliding with the pedestal.

USDCraftLoading 3D

Manual pallet jack

Text → asset

Prompt

Create a complete, functionally usable, simulator-ready manual pallet jack with pivoting steering handle, rotating steer and load wheels, lifting fork mechanism, control lever, and clear pallet entry.

USDCraftLoading 3D

Explore the generated assets

16 categories · pick an asset, scrub its joints, download the USDZ

Drag to orbit · scroll to zoom

Input image credits

Domestic sewing machine — eggrole, CC BY 2.0
Glass infuser teapot — USDCraft evaluation set
Microwave oven — USDCraft evaluation set
Reflective traffic cone — Pryskyřník, Public domain
Upright piano — pianomap, CC BY 2.0
Backyard playground set — USDCraft evaluation set
Camera tripod — David Adam Kess, CC BY-SA 4.0
Cubby bookshelf with drawers — USDCraft evaluation set
Emergency exit sign — frozenchipmunk, CC BY 2.0
Laboratory centrifuge — Ivangiesen, CC0 1.0
Parcel locker bank — PinTweaks, CC0 1.0
Potted houseplant — Thomas Tolkien, CC BY 2.0
Stocked supermarket shelf — USDCraft evaluation set

Reconstruction

Rebuilding real objects for simulation

For real-to-sim, USDCraft rebuilds each source mesh as a program, separating fused parts and completing missing ones. The input can be a static asset, a raw phone scan, or a single image lifted to a mesh.

Source meshLoading 3D

USDCraft–AstraLoading 3D

Views are linked — orbit either side to compare the source with the rebuilt, articulated asset.

Comparison with prior methods

Same Hunyuan3D 2.1 input for every method · paper results (USDCraft and Mini Workflow: round 1)

Loading 3D

Input image of the ergonomic office chair

Input · image + Hunyuan3D 2.1 mesh

No complete articulated outputpart geometry missing

ArtLLM

Loading 3D

SIMART3 parts · 2 joints

Loading 3D

Particulate9 parts · 8 joints

Loading 3D

Mini Workflow–Astra12 parts · 11 joints

Loading 3D

USDCraft–Astra15 parts · 14 joints

All six views share one camera — part colors show how each method splits the object into rigid parts; joints sweep through their limits.

Simulation

Loads into Isaac Sim, ready to run

Every compiled asset carries collision geometry, masses and joint limits, so it runs in Isaac Sim (PhysX) without manual fixes. Each clip shows the static asset, its dynamic articulation, and its collision geometry.

Warehouse forklift

Compact excavator

Baby grand piano

Robotic workbench arm

Real2Sim2Real

Policies trained only in simulation, deployed on real objects

We scan three real objects, rebuild them with USDCraft, collect 200 demonstrations per task in simulation, and train a Diffusion Policy that is deployed on the physical robot with no real-world data.

Open drawer

Simulation · data collection

Real world · policy rollout (4× speed)

Press toaster lever

Simulation · data collection

Real world · policy rollout (4× speed)

Turn on toaster

Simulation · policy rollout

Real world · policy rollout (4× speed)

Success rate across asset methods

Paper Table · 20 trials per setting · ↓ sim-to-real drop · I/M: image/mesh input

Asset method	Input	Open drawer		Press toaster lever		Turn on toaster
Asset method	Input	Sim	Real	Sim	Real	Sim	Real
Articraft–Sol	I	90%	20%↓70	90%	10%↓80	80%	20%↓60
Particulate	M	0%	0%	50%	10%↓40	0%	0%
Mini Workflow–Astra	I+M	100%	0%↓100	90%	60%↓30	85%	70%↓15
USDCraft–Astra	I+M	100%	90%↓10	90%	85%↓5	85%	85%↓0

BibTeX

@article{zhang2026usdcraft,
  title={USDCraft: Geometrically Grounded Programmatic Modeling of Articulated 3D Assets for Simulation},
  author={Zhang, Chuanrui and Yang, Zaijia and Wang, Duomin and Shi, Lu and Zhou, Daquan and Zhang, Ruihua and Wang, Ziwei},
  journal={arXiv preprint arXiv:2610.11322},
  year={2026}
}