*********************************
There is now a CONTENT FREEZE for Mercury while we switch to a new platform. It began on Friday, March 10 at 6pm and will end on Wednesday, March 15 at noon. No new content can be created during this time, but all material in the system as of the beginning of the freeze will be migrated to the new platform, including users and groups. Functionally the new site is identical to the old one. webteam@gatech.edu
*********************************
Title: Towards Realistic Embodied AI Agents
Date: Friday, July 1st, 2022
Time: 5-7pm (ET)
Location (virtual): https://gatech.zoom.us/j/4936331597?pwd=RVp2TUVsYjM2ZGxsay9sKzJNN25FZz09
Samyak Datta
School of Interactive Computing
College of Computing
Georgia Institute of Technology
Committee:
Dr. Devi Parikh (advisor), School of Interactive Computing, Georgia Institute of Technology
Dr. Dhruv Batra, School of Interactive Computing, Georgia Institute of Technology
Dr. Judy Hoffman, School of Interactive Computing, Georgia Institute of Technology
Dr. Roozbeh Mottaghi, Allen Institute for AI (AI2)
Dr. Peter Anderson, Google
Abstract:
Recent years has witnessed the inception of a growing field of inquiry within the broader AI community termed as "Embodied AI". Problems studied under the umbrella of Embodied AI include the introduction of scene datasets and simulators to train AI agents to perform a wide spectrum of tasks requiring a curriculum of capabilities. While progress on this front has been commendable, it is nonetheless important and worthwhile to pause and carefully examine the real-world context under which such AI agents would be expected to operate. While doing so, it is critical to ensure "realism" i.e. the settings, parameters, and assumptions under which these agents and tasks are investigated in simulation indeed serve as the right test beds and high-fidelity precursors to the real world. Simulation has its own advantages of being fast, scalable/distributed, and safe and therefore, it is valuable to strive to make simulations more realistic.
Towards that end, this thesis serves as an investigation into realism for Embodied AI agents in simulation. We study realism along 3 different axes. (1) Photorealism: The visual appearance of objects and rooms in indoor scenes, as viewed by the agent in simulation, must be a close approximation of what the agent would actually see in the real world. (2) Sensing and Actuation Realism: Embodied agents in simulation are often equipped with a variety of idealized sensors that provide highly privileged, noise-free sensing signals, depending on the task they are being trained for and take deterministic actions. This is in contrast to the dirty reality of noisy sensors and actuations in the real world. (3) Task Realism: Moving beyond realistic sensors and actuations, we need to ensure that the assumptions made while formulating tasks and the settings under which these tasks are being evaluated in simulation does indeed bode well with the deployment scenarios and use-cases in the real world. Finally, we also explore connections between these different axes of realism.
