Modern Statistical Theory Inspired by Deep Learning

*********************************
There is now a CONTENT FREEZE for Mercury while we switch to a new platform. It began on Friday, March 10 at 6pm and will end on Wednesday, March 15 at noon. No new content can be created during this time, but all material in the system as of the beginning of the freeze will be migrated to the new platform, including users and groups. Functionally the new site is identical to the old one. webteam@gatech.edu
*********************************

Event Details
  • Date/Time:
    • Wednesday March 6, 2019
      1:30 pm - 2:30 pm
  • Location: ISyE Main 228
  • Phone:
  • URL: Groseclose Building
  • Email:
  • Fee(s):
    N/A
  • Extras:
Contact
No contact information submitted.
Summaries

Summary Sentence: Modern Statistical Theory Inspired by Deep Learning

Full Summary: Abstract: Modern learning algorithms, such as deep learning, have gained great successes in real applications. However, some of their empirical behaviors may not be interpreted within the classical statistical learning framework. For example, deep learning algorithms achieve small testing error even when the training error is zero, i.e., over-fitting. Another phenomenon is observed in image recognition applications where a hardly noticeable change of data may lead to dramatic increase of mis-classification rates. Inspired by these observations, we attempt to illustrate new theoretical insights for data-interpolation and adversarial testing using the very simple nearest neighbor algorithms. In particular, we prove statistical optimality of interpolated nearest neighbor algorithms. More surprisingly, it is discovered that the classification performance, under a proper interpolation, is even better that the best kNN in terms of multiplicative constant. As for adversarial testing, we demonstrate that different adversarial mechanisms lead to different phase transition phenomena of mis-classification rate in terms of its upper bound. Additionally, our technical analysis invented to deal with adversarial samples can also be applied to other variants of kNN, e.g. pre-processed 1NN and distributed-NN.   Bio: Guang Cheng is a Professor of Statistics at Purdue University.  He received his PhD in Statistics from University of Wisconsin-Madison in 2006.  His research interests include Big Data and High Dimensional Statistical Inferences, and more recently turn to Deep Learning and Reinforcement Learning.  Cheng is the recipient of the NSF CAREER award, Noether Young Scholar Award and Simons Fellowship in Mathematics. Please visit his big data theory research group at http://www.science.purdue.edu/bigdata/

Title: Modern Statistical Theory Inspired by Deep Learning

Abstract: Modern learning algorithms, such as deep learning, have gained great successes in real applications. However, some of their empirical behaviors may not be interpreted within the classical statistical learning framework. For example, deep learning algorithms achieve small testing error even when the training error is zero, i.e., over-fitting. Another phenomenon is observed in image recognition applications where a hardly noticeable change of data may lead to a dramatic increase in misclassification rates. Inspired by these observations, we attempt to illustrate new theoretical insights for data-interpolation and adversarial testing using the very simple nearest neighbor algorithms. In particular, we prove statistical optimality of interpolated nearest neighbor algorithms. More surprisingly, it is discovered that the classification performance, under a proper interpolation, is even better than the best kNN in terms of multiplicative constant. As for adversarial testing, we demonstrate that different adversarial mechanisms lead to different phase transition phenomena of the misclassification rate in terms of its upper bound. Additionally, our technical analysis invented to deal with adversarial samples can also be applied to other variants of kNN, e.g. pre-processed 1NN and distributed-NN.

 

Bio: Guang Cheng is a Professor of Statistics at Purdue University.  He received his Ph.D. in Statistics from the University of Wisconsin-Madison in 2006.  His research interests include Big Data and High Dimensional Statistical Inferences, and more recently turned to Deep Learning and Reinforcement Learning.  Cheng is the recipient of the NSF CAREER award, Noether Young Scholar Award and Simons Fellowship in Mathematics. Please visit his big data theory research group at http://www.science.purdue.edu/bigdata/

 

Additional Information

In Campus Calendar
Yes
Groups

TRIAD

Invited Audience
Faculty/Staff, Postdoc, Public, Graduate students
Categories
Seminar/Lecture/Colloquium
Keywords
deep learning
Status
  • Created By: Xiaoming Huo
  • Workflow Status: Published
  • Created On: Mar 3, 2019 - 8:52pm
  • Last Updated: Mar 3, 2019 - 8:58pm