Interactive System – 3D motion Capture + Interactive Machine Learning

Collaboration with Jules Francoise and Frederic Bevilacqua (IRCAM), Thecla Schiphorst,(SIAT – SFU), Karen Studd (LIMS)

Human movement has historically been approached as a functional component of interaction within HCI. Yet movement is not only functional, it is also highly experiential. In this research project, we explore how movement expertise as articulated in Laban Movement Analysis can contribute to the design of computational models of movement’s expressive qualities as defined in the framework of Laban Effort theory. We included experts in LMA in our design process, in order to select a set of suitable multimodal sensors including Vicon Motion Capture, Kinect, accelerometers and electromyograms. We also computed high–level features that closely correlate to the definitions of Efforts in LMA. We evaluated the selected sensors and the high–level features for the recognition of the Weight, Time and Space Efforts using a Machine Learning Algorithm based on Hierarchical Hidden Markov Models. Our results showed that the optimal Weight and Time Effort recognition was achieved by the high–level features informed by LMA expertise applied to multimodal sensors of accelerometers and electromyograms.

Publication: Sarah Fdili Alaoui, Jules Françoise, Thecla Schiphorst, Karen Studd, Fréderic Bevilacqua “Seeing, Sensing and Recognizing Laban Movement Qualities”, In Proceedings of ACM Conference on Human Factors in Computing Systems (CHI), Denver, 2017.