Phd Defense by Jianwei Yang

*********************************
There is now a CONTENT FREEZE for Mercury while we switch to a new platform. It began on Friday, March 10 at 6pm and will end on Wednesday, March 15 at noon. No new content can be created during this time, but all material in the system as of the beginning of the freeze will be migrated to the new platform, including users and groups. Functionally the new site is identical to the old one. webteam@gatech.edu
*********************************

Event Details
  • Date/Time:
    • Thursday January 2, 2020 - Friday January 3, 2020
      4:00 pm - 5:59 pm
  • Location: Coda C1003 Adair
  • Phone:
  • URL:
  • Email:
  • Fee(s):
    N/A
  • Extras:
Contact
No contact information submitted.
Summaries

Summary Sentence: Structured Visual Understanding, Generation and Reasoning

Full Summary: No summary paragraph submitted.

Title: Structured Visual Understanding, Generation and Reasoning

 

Jianwei Yang

Ph.D. Candidate in Computer Science

School of Interactive Computing

Georgia Institute of Technology

https://www.cc.gatech.edu/~jyang375/

 

Date: Thursday, January 2nd, 2020

Time: 4:00-6:00 PM (EST)

Location: Coda C1003 Adair

BlueJeans: https://bluejeans.com/998872971

 

 

Committee:

Dr. Devi Parikh (Advisor), School of Interactive Computing, Georgia Institute of Technology

Dr. Dhruv Batra, School of Interactive Computing, Georgia Institute of Technology

Dr. David Crandall, School of Informatics, Computing and Engineering, Indiana University

Dr. Stefan Lee, School of Electrical  Engineering and Computer Science, Oregon State University

Dr. Judy Hoffman, School of Interactive Computing, Georgia Institute of Technology

 

Abstract:

 

The world around us is highly structured. In the real world, multiple objects usually exist in a scene and interact with each other in predictable ways (e.g., mug on table, keyboard below computer monitor); for a single object, it usually consists of multiple components under some structured configurations (e.g., a person has different body parts). These structures manifest themselves in the visual data that captures the world around us, and thus can potentially provide a strong inductive bias to various vision tasks. In this talk, I will discuss how to integrate such structure priors into different tasks including visual understanding, generation and reasoning. Specifically,

 

  1. I will talk about how to integrate the structure prior into visual system to improve its visual understanding ability at different levels. Particularly, I will talk about how we leverage the relational sparsity and context in images for a better scene graph generation;
  2. I will show how we can exploit the structure in images to address the dual problem, visual generation. Specifically, I will explain how we can generate images in a compositional manner by generating background and foreground objects separately and compose them together;
  3. I will discuss how to use the structured semantic visual representations as the interface to bridge visual perception and reasoning to address vision and language tasks. Specifically, I will present a meta-active-learning framework in which the agent reasonings on symbolic scene graph to generate informative questions to ask oracle to improve its visual system incrementally.

 

On these different levels of tasks, we demonstrate that modeling the structures in visual data and the associated text can not only improve the model performance but also increase the model transparency. To the end, I will briefly discuss the challenges in this domain and the extensions of recent works.

Additional Information

In Campus Calendar
No
Groups

Graduate Studies

Invited Audience
Faculty/Staff, Public, Graduate students, Undergraduate students
Categories
Other/Miscellaneous
Keywords
Phd Defense
Status
  • Created By: Tatianna Richardson
  • Workflow Status: Published
  • Created On: Jan 2, 2020 - 1:54pm
  • Last Updated: Jan 2, 2020 - 1:54pm