*********************************
There is now a CONTENT FREEZE for Mercury while we switch to a new platform. It began on Friday, March 10 at 6pm and will end on Wednesday, March 15 at noon. No new content can be created during this time, but all material in the system as of the beginning of the freeze will be migrated to the new platform, including users and groups. Functionally the new site is identical to the old one. webteam@gatech.edu
*********************************
Title: Improving Vision-based Robotic Manipulation with Affordance Understanding
Committee:
Dr. Patricio Vela, ECE, Chair , Advisor
Dr. Anthony Yezzi, ECE
Dr. Ghassan AlRegib, ECE
Dr. Sonia Chernova, CoC
Dr. Eva Dyer, BME
Abstract: The objective of the thesis is to improve robotic manipulation via vision-based affordance understanding, which would advance the application of robotics to industrial use cases, as well as impact the area of assistive robotics. In this thesis, we focus on vision-based manipulation for real-time robotic application. A series of methods are proposed to improve the applicability for practical scenarios. Specifically we tackle the problem of identifying viable candidate robotic grasps of objects, and seek for more general affordance map prediction methods with reduced annotation costs. Besides, we target on generalizing learned affordances to unseen categories, and predicting multiple ranked affordance for each object part. We aim to narrow the bridge between the vision detection to robotic manipulations by linking action primitives to task execution in real-world. To account for various shapes and poses of objects for universal grasp identification, CNN-based architecture is adopted to learn grasp representation without hand-engineering features. Unlike regression methods, the identification of grasp configurations in this architecture is broken into a grasp detection process, followed by a more refined grasp orientation classification process, where both processes are embedded within two coupled networks. To reduce the labor-intensive annotation cost, learning from supervised synthetic data with unlabeled real images is considered. To maintain the advantage of jointly optimizing detection and affordance prediction, labeled synthetic data is applied and jointly adapted to unlabelled real images for detection and affordance segmentation. To preserve the advantages of an object-based method while generalizing to unseen categories, binary classification mode is added for objectness detection and localization. The proposed architecture further adopts KL-divergence to learn the distributions instead of cross entropy for a single label ground truth on each pixel,enabling multiple ranked affordance prediction of one object part. Improvements on affordance prediction is made by proposed branch-wise attention module and attribute-like auxiliary task. A system combining proposed affordance detector with a pre-trained object detector illustrates the usage with the Planning Domain Definition Language (PDDL) in practical robotic manipulation applications.
