“Predicting Co-verbal Gestures: A Deep and Temporal Modeling Approach” by Chung-Cheng Chiu, Louis-Philippe Morency, and Stacy Marsella. In Intelligent Virtual Agents: 15th International Conference, IVA 2015, Delft, The Netherlands, August 26-28, 2015, Proceedings, (Willem-Paul Brinkman, Joost Broekens, and Dirk Heylen, eds.), 2015, pp. 152-166.
Gestures during spoken dialog play a central role in human communication. As a consequence, models of gesture generation are a key challenge in research on virtual humans, embodied agents capable of face-to-face interaction with people. Machine learning approaches to gesture generation must take into account the conceptual content in utterances, physical properties of speech signals and the physical properties of the gestures themselves. To address this challenge, we proposed a gestural sign scheme to facilitate supervised learning and presented the DCNF model, a model to jointly learn deep neural networks and second order linear chain temporal contingency. The approach we took realizes both the mapping relation between speech and gestures while taking account temporal relations among gestures. Our experiments on human co-verbal dataset shows significant improvement over previous work on gesture prediction. A generalization experiment performed on handwriting recognition also shows that DCNFs outperform the state-of-the-art approaches.
BibTeX entry:
@incollection{ChiuIVA15,
author = {Chung-Cheng Chiu and Louis-Philippe Morency and Stacy Marsella},
editor = {Willem-Paul Brinkman and Joost Broekens and Dirk Heylen},
title = {Predicting Co-verbal Gestures: A Deep and Temporal Modeling
Approach},
booktitle = {Intelligent Virtual Agents: 15th International Conference,
IVA 2015, Delft, The Netherlands, August 26-28, 2015,
Proceedings},
pages = {152--166},
publisher = {Springer International Publishing},
year = {2015},
url = {https://stacymarsella.org/publications/pdf/ChiuIVA15.pdf}
}
(This webpage was created with bibtex2web.)
Back to Stacy Marsella.