Schmidt, Dominik (2007) Acceleration-Based Gesture Recognition for Conducting with Hidden Markov Models. Masters thesis, UNSPECIFIED.
2007_-_Schmidt,_D_-_Acceleration-Based_Gesture_Recognition_for_Conducting_with_Hidden_Markov_Models.pdf
Download (23MB)
Abstract
This thesis focuses on developing new means of input for an existent system allowing users with any skill to conduct a virtual orchestra in realtime. As a first step, several user studies with musical conductors were performed to learn more about the interaction between conductor and orchestra. The eWatch, a wearable computing and sensing platform, is selected as input device. It is worn like a regular wrist watch and transmits acceleration data via Bluetooth to the host computer. Hidden Markov models are chosen to perform the gesture recognition task. While realizing a reliable and continuous gesture recognition, the proposed system is able to successfully reject non-meaningful hand movements, applying a threshold model. The system makes use of a discrete hidden Markov model in conjunction with a modified version of the Viterbi algorithm, which is specifically adopted for continuous gesture recognition. By performing common conducting gestures, the user is able to indicate the measure. In addition, he or she can influence the orchestra’s tempo and volume by varying the gestures in speed and size. The evaluation of the proposed approach with different users yields a detection ratio of more than 94% and a reliability of more than 90% with an average latency around 90 ms for the user-dependent recognition. While focusing on the application domain of conducting, the proposed recognition approach is versatility applicable to arbitrary user interfaces benefiting from gesture input. Using a machine learning approach, new gestures can easily be trained.