*********************************
There is now a CONTENT FREEZE for Mercury while we switch to a new platform. It began on Friday, March 10 at 6pm and will end on Wednesday, March 15 at noon. No new content can be created during this time, but all material in the system as of the beginning of the freeze will be migrated to the new platform, including users and groups. Functionally the new site is identical to the old one. webteam@gatech.edu
*********************************
Title: Grounded Semantic Reasoning for Interacting with Real-World Objects
Date: Monday, November 14th, 2022
Time: 10:00 AM – 12:00 PM EST
Location: Zoom meeting (https://gatech.zoom.us/j/9702258427)
Weiyu Liu
Robotics Ph.D. Student
School of Interactive Computing
Georgia Institute of Technology
Committee:
Dr. Sonia Chernova (Advisor) – School of Interactive Computing, Georgia Institute of Technology
Dr. Charlie Kemp – Department of Biomedical Engineering, Georgia Institute of Technology
Dr. Jesse Thomason – Department of Computer Science, University of Southern California
Dr. Animesh Garg – Department of Computer Science, University of Toronto
Dr. Chad Jenkins – Department of Electrical Engineering and Computer Science, University of Michigan
Abstract:
To operate in unstructured environments such as homes and offices, robots need to manipulate novel objects while adapting to changes in environments and goals. Semantic knowledge about objects, which represent relations between object categories, locations, properties, and uses, can reveal meaningful connections between problems and environments. However, closely integrating semantic knowledge and sensorimotor data (e.g., object point clouds, 6-DoF poses, and attributes detected with multimodal sensing) remains a key challenge because the two types of data have drastically different characteristics in terms of modality, complexity, and levels of abstraction. This thesis develops semantic reasoning frameworks capable of modeling complex semantic knowledge grounded in robot perception and action. We show that grounded semantic reasoning enables robots to more effectively perceive, model, and manipulate objects in real-world environments. Specifically, this thesis makes the following contributions: 1) a survey of semantic reasoning providing a unified view for the diversity of works in the field, 2) a method for predicting missing relations in large-scale knowledge graphs by leveraging type hierarchies of entities, 3) an n-ary knowledge representation for predicting unknown properties of objects in various environmental contexts, 4) semantic grasping methods that account for a broad sense of contexts and achieve generalization by reasoning about semantics of tasks and objects, 5) methods for rearranging novel objects into semantically meaningful spatial structures based on high-level language instructions.
