Model-based perception and planning let robots grasp objects reliably in structured settings, but learned approaches can fail on a simple block-stacking task once conditions get even slightly more realistic. Existing manipulation datasets also didn’t capture end-to-end task planning with obstacle avoidance.
The CoSTAR Block Stacking Dataset lets researchers study how learning systems handle workspace constraints, using a robot that grasps and stacks colored blocks.
My part: I wrote the data loader, training scripts and visualization tools, packaged them on PyPI for TensorFlow and PyTorch, and added documentation, examples and training splits. Batch loading got up to 80% faster, which cut a 200-epoch training run from 800+ hours to about 200.
A. Hundt, V. Jain, C. H. Lin, C. Paxton, and G. D. Hager, “The CoSTAR Block Stacking Dataset: Learning with Workspace Constraints,” IROS 2019.