SCDP
Precise Manipulation with a Single Camera
Spatially Conditioned Diffusion Policy learns precise manipulation from a single global-view RGB camera, with visual anchors that reveal where the policy looks.
CoRL · 2026 · Accepted
Spatial Conditioning with Visual Attention Anchors
A single global camera must capture both the workspace and the small visual details needed for precise manipulation. SCDP addresses this challenge by using intermediate end-effector trajectories as visual attention anchors. At each diffusion step, these trajectories identify locations in the image where the policy can gather task-relevant features.
- Multi-scale image encoding. The visual encoder produces feature maps at several scales, retaining both broad scene context and fine visual detail.
- Spatial conditioning. Reconstructed end-effector trajectories are projected into the image. The model samples point-wise features from the multi-scale maps at these visual attention anchors.
- Conditional action denoising. The sampled spatial context conditions the action denoising network through FiLM and is updated at each denoising step.
The implementation is available in the SCDP code repository.
Simulation Results
SCDP achieves more than 80% success in each Meta-World difficulty group using a single RGB camera. The comparison below reports the success rates shown on the official project page: DP uses RGB observations, while DP3 uses depth observations.
Single-View Performance on Hard Tasks
| Method | Camera setting | Success (%) |
|---|---|---|
| DP + Wrist | RGB with a wrist view | 74.2 ± 29.4 |
| DP3 | Depth | 60.3 ± 38.0 |
| SCDP | Single RGB | 82.5 ± 11.5 |
Real-World Manipulation
The following autonomous demonstrations use a single global-view RGB camera. They cover precise insertion, visual distractors, sequential manipulation, and different grasp geometries. Captions retain the playback speeds of the original demonstrations.
Precise Manipulation
USB insertion requires grasping the drive, aligning it with the hub port, and pushing it into place. Battery insertion additionally requires overcoming spring resistance to seat the battery in its slot.
Robustness to Visual Distractors
The insertion tasks are repeated with additional objects in the workspace, testing whether the policy attends to the relevant interaction.
Long-Horizon Manipulation
The drawer task links four stages: open the drawer, pick up a banana, place it inside, and close the drawer.
Cup Grasping and Cube Pushing
Further tasks require adapting a grasp to the cup handle, placing one finger inside a cup to grasp its wall without hitting the rim or handle, and pushing a target cube among distractors.
Humanoid Demonstrations
Additional demonstrations apply the same approach to a humanoid using a single head-mounted camera for egocentric visual observation. The robot separates paper cups on a tabletop and uses an eraser to wipe a whiteboard.
Representative Publications
Spatially Conditioned Diffusion Policy: Learning Precise and Robust Manipulation with a Single RGB Camera
Seoyoon Kim, Kanghyun Kim, Dongwoo Ko, Yeong Jin Heo, and Min Jun Kim
Conference on Robot Learning (CoRL), 2026, Accepted
Bib
@inproceedings{kim2026spatially,
title={Spatially Conditioned Diffusion Policy: Learning Precise and Robust Manipulation with a Single RGB Camera},
author={Kim, Seoyoon and Kim, Kanghyun and Ko, Dongwoo and Heo, Yeong Jin and Kim, Min Jun},
booktitle={Conference on Robot Learning (CoRL)},
year={2026},
note={Accepted},
eprint={2606.14535},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.14535}
}Figures, demonstrations, and research details: author's project page.
Explore more in Physical AI