*********************************
There is now a CONTENT FREEZE for Mercury while we switch to a new platform. It began on Friday, March 10 at 6pm and will end on Wednesday, March 15 at noon. No new content can be created during this time, but all material in the system as of the beginning of the freeze will be migrated to the new platform, including users and groups. Functionally the new site is identical to the old one. webteam@gatech.edu
*********************************
Title: A Feedback Methodology for Task-Driven Fine-Grained Pixel Control in Smart Cameras
Committee:
Dr. Mukhopadhyay, Advisor
Dr. Chatterjee, Chair
Dr. Krishna
Abstract:
The objective of the proposed research is to develop the concept of feedback control for smart camera systems driven by an end-task. Specifically, this camera system only captures useful information pertaining to an end-user defined task and at the highest quality. This is achieved through task-guided feedback provided by an embedded deep-neural network based task. The advantage of feedback is shown at the algorithm level, at the encoder level and at the level of the sensor itself. The feedback modifies the sampling characteristics of each individual pixel to achieve what is essentially task-guided information modulation. A number of control schemes and tasks are presented including spatial resolution control and temporal resolution control. Multiple use cases including object detection, object tracking, action detection, anomaly detection are demonstrated and their associated control schemes. Experiments show that the proposed schemes preserve or increase task accuracy while utilizing 3-4x lower bandwidth. Additionally, it is shown that for the feedback system, latency is a unique challenge particularly due to the high computational complexity of DNN networks. A simulation model for the feedback is developed to show how software/hardware solutions can be leveraged to deal with the latency challenge. Further, some qualitative challenges are presented encountered in unconstrained videos including small object detection and action detection in moving camera sequences. The remainder of the research will focus on developing an edge-host collaborative methodology and fast uncertainty estimation in the smart camera.
