*********************************
There is now a CONTENT FREEZE for Mercury while we switch to a new platform. It began on Friday, March 10 at 6pm and will end on Wednesday, March 15 at noon. No new content can be created during this time, but all material in the system as of the beginning of the freeze will be migrated to the new platform, including users and groups. Functionally the new site is identical to the old one. webteam@gatech.edu
*********************************
Ph.D. Thesis Proposal Announcement
Title: Recognition of American Sign Language Classifiers
Zahoor Zafrulla
Computer Science Ph.D. Student
School of Interactive Computing
College of Computing
Georgia Institute of Technology
Date: December 14, 2012 (Friday)
Time: 12p - 2p EST
Location: TBD
Committee:
Abstract:
In this proposal I address the problem of automatically recognizing selected classifier-based grammatical structures of American Sign Language (ASL). Classifiers in ASL utilize surrogate handshapes for people or objects and provide information about their location, movement and appearance. In the past researchers have focused on recognition of finger spelling, isolated signs, facial expressions and WH-questions. Challenging problems such as recognition of sign-based grammatical structures, which consist of basic signs strung together to form phrases or sentences, and classifier-based grammatical structures are relatively unexplored in the field of ASL recognition.
In our work we have developed CopyCat, an educational ASL game, that was designed to help deaf children improve their language abilities. CopyCat requires children to engage in a progressively more difficult expressive signing task, to describe a graphic, as they advance through the game.
I will show that by leveraging context we can use verification, in place of recognition, to boost machine performance for determining if the signed responses in an expressive signing task, like in the CopyCat game, are correct or incorrect. I propose to improve the quality of the machine verifier’s ability to identify the boundary of the signs by using a novel two-pass technique that combines signed input in both forward and reverse directions. Additionally, I will reduce CopyCat’s dependency on custom manufactured hardware by using an off-the-shelf Kinect camera to achieve similar verification performance. I propose that we can extend our ability to recognize sign language by leveraging depth maps and creating an architecture to recognize selected classifier-based grammatical structures of ASL. I will also demonstrate the flexibility of the architecture by showing that the architecture is able to spot and recognize ASL classifier constructions that are embedded within an ASL narrative.
