Fully-Adaptive RF Sensing for Non-Intrusive ASL Recognition via Interactive Smart Environments

The past decade has seen great advancements in speech recognition for control of interactivedevices, personal assistants, and computer interfaces. However, Deaf people andpeople with hard-of-hearing, whose primary mode of communication is sign language, cannotuse voice-controlled interfaces. Although there has been significant work in video-basedsign language recognition, video is not effective in the dark and has raised privacy concernsin the Deaf community when used in the context of human ambient intelligence. Radarshave recently been started to be used as a new modality that can be effective under thecircumstances where video is not. This dissertation conducts a thorough exploration of the challenges in RF-enabled signlanguage recognition systems. Specifically, it proposes an end-to-end framework to acquire,temporally isolate, and recognize individual signs. A trigger sign detection with an adaptivethresholding method is also proposed. An angular subspace projection method is presentedto separate multiple targets at raw data level. An interactive sign language-controlled chessgame is designed to enhance the user experience and automate the data collection andannotation process for labor-intensive data collection procedure. Finally, a framework ispresented to dynamically adjust radar waveform parameters based on human presence andtheir activity.