2016 |
Zhang, Zheng; Girard, Jeff M; Wu, Yue; Zhang, Xing; Liu, Peng; Ciftci, Umur; Canavan, Shaun; Reale, Michael; Horowitz, Andy; Yang, Huiyuan; Cohn, Jeffrey F; Ji, Qiang; Yin, Lijun Multimodal spontaneous emotion corpus for human behavior analysis Conference Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2016, 2016. Abstract | Links | BibTeX | Tags: 3D facial expression, Spontaneous expression @conference{Zhang2016, title = {Multimodal spontaneous emotion corpus for human behavior analysis}, author = {Zheng Zhang and Jeff M Girard and Yue Wu and Xing Zhang and Peng Liu and Umur Ciftci and Shaun Canavan and Michael Reale and Andy Horowitz and Huiyuan Yang and Jeffrey F Cohn and Qiang Ji and Lijun Yin}, url = {http://www.cv-foundation.org/openaccess/content_cvpr_2016/html/Zhang_Multimodal_Spontaneous_Emotion_CVPR_2016_paper.html}, year = {2016}, date = {2016-06-30}, booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2016}, pages = {3438-3446}, abstract = {Emotion is expressed in multiple modalities, yet most research has considered at most one or two. This stems in part from the lack of large, diverse, well-annotated, multimodal databases with which to develop and test algorithms. We present a well-annotated, multimodal, multidimensional spontaneous emotion corpus of 140 participants. Emotion inductions were highly varied. Data were acquired from a variety of sensors of the face that included high-resolution 3D dynamic imaging, high-resolution 2D video, and thermal (infrared) sensing, and contact physiological sensors that included electrical conductivity of the skin, respiration, blood pressure, and heart rate. Facial expression was annotated for both the occurrence and intensity of facial action units from 2D video by experts in the Facial Action Coding System (FACS). The corpus further includes derived features from 3D, 2D, and IR (infrared) sensors and baseline results for facial expression and action unit detection. The entire corpus will be made available to the research community.}, keywords = {3D facial expression, Spontaneous expression}, pubstate = {published}, tppubtype = {conference} } Emotion is expressed in multiple modalities, yet most research has considered at most one or two. This stems in part from the lack of large, diverse, well-annotated, multimodal databases with which to develop and test algorithms. We present a well-annotated, multimodal, multidimensional spontaneous emotion corpus of 140 participants. Emotion inductions were highly varied. Data were acquired from a variety of sensors of the face that included high-resolution 3D dynamic imaging, high-resolution 2D video, and thermal (infrared) sensing, and contact physiological sensors that included electrical conductivity of the skin, respiration, blood pressure, and heart rate. Facial expression was annotated for both the occurrence and intensity of facial action units from 2D video by experts in the Facial Action Coding System (FACS). The corpus further includes derived features from 3D, 2D, and IR (infrared) sensors and baseline results for facial expression and action unit detection. The entire corpus will be made available to the research community. |
2015 |
Zhang, Xing; Ciftci, Umur A; Yin, Lijun Mouth Gesture based Emotion Awareness and Interaction in Virtual Reality Conference SIGGRAPH '15 ACM SIGGRAPH 2015 Posters, (26), SIGGRAPH ACM Digital Library, 2015, ISBN: 978-1-4503-3632-1 . Abstract | Links | BibTeX | Tags: 3D facial expression, GPU, Virtual reality @conference{Zhang2015, title = {Mouth Gesture based Emotion Awareness and Interaction in Virtual Reality}, author = {Xing Zhang and Umur A Ciftci and Lijun Yin}, doi = {10.1145/2787626.2787635}, isbn = {978-1-4503-3632-1 }, year = {2015}, date = {2015-07-14}, booktitle = {SIGGRAPH '15 ACM SIGGRAPH 2015 Posters}, number = {26}, publisher = {ACM Digital Library}, organization = {SIGGRAPH}, abstract = {In recent years, Virtual Reality (VR) has become a new media to provide users an immersive experience. Events happening in the VR connect closer to our emotions as compared to other interfaces. The emotion variations are reflected as our facial expressions. However, the current VR systems concentrate on "giving" information to the user, yet ignore "receiving" emotional status from the user, while this information definitely contributes to the media content rating and the user experience. On the other hand, traditional controllers become difficult to use due to the obscured view point. Hand and head gesture based control is an option [Cruz-Neira et al. 1993]. However, certain sensor devices need to be worn to assure control accuracy and users are easy to feel tired. Although face tracking achieves accurate result in both 2D and 3D scenarios, the current state-of-the-art systems cannot work when half of the face is occluded by the VR headset because the shape model is trained by data from the whole face.}, keywords = {3D facial expression, GPU, Virtual reality}, pubstate = {published}, tppubtype = {conference} } In recent years, Virtual Reality (VR) has become a new media to provide users an immersive experience. Events happening in the VR connect closer to our emotions as compared to other interfaces. The emotion variations are reflected as our facial expressions. However, the current VR systems concentrate on "giving" information to the user, yet ignore "receiving" emotional status from the user, while this information definitely contributes to the media content rating and the user experience. On the other hand, traditional controllers become difficult to use due to the obscured view point. Hand and head gesture based control is an option [Cruz-Neira et al. 1993]. However, certain sensor devices need to be worn to assure control accuracy and users are easy to feel tired. Although face tracking achieves accurate result in both 2D and 3D scenarios, the current state-of-the-art systems cannot work when half of the face is occluded by the VR headset because the shape model is trained by data from the whole face. |
2014 |
Zhang, Xing; Yin, Lijun; Cohn, Jeffrey F; Canavan, Shaun; Reale, Michael; Horowitz, Andy; Liu, Peng; Girard, Jeffrey M BP4D-Spontaneous: a high-resolution spontaneous 3D dynamic facial expression database Journal Article Image and Vision Computing, 32 (10), pp. 692-706, 2014. Abstract | Links | BibTeX | Tags: 3D facial expression, Dynamic facial expression database, FACS, Spontaneous expression @article{Zhang2014, title = {BP4D-Spontaneous: a high-resolution spontaneous 3D dynamic facial expression database}, author = {Xing Zhang and Lijun Yin and Jeffrey F. Cohn and Shaun Canavan and Michael Reale and Andy Horowitz and Peng Liu and Jeffrey M. Girard}, url = {http://www.sciencedirect.com/science/article/pii/S0262885614001012}, doi = {10.1016/j.imavis.2014.06.002}, year = {2014}, date = {2014-07-14}, journal = {Image and Vision Computing}, volume = {32}, number = {10}, pages = {692-706}, abstract = {Facial expression is central to human experience. Its efficiency and valid measurement are challenges that automated facial image analysis seeks to address. Most publically available databases are limited to 2D static images or video of posed facial behavior. Because posed and un-posed (aka “spontaneous”) facial expressions differ along several dimensions including complexity and timing,well-annotated video of un-posed facial behavior is needed. Moreover, because the face is a three-dimensional deformable object, 2D video may be insufficient, and therefore 3D video archives are required.We present a newly developed 3D video database of spontaneous facial expressions in a diverse group of young adults. Well-validated emotion inductions were used to elicit expressions of emotion and paralinguistic communication. Frame-level ground-truth for facial actions was obtained using the Facial Action Coding System. Facial features were tracked in both 2D and 3D domains. To the best of our knowledge, this newdatabase is the first of its kind for the public. Thework promotes the exploration of 3D spatiotemporal features in subtle facial expression, better understanding of the relation between pose and motion dynamics in facial action units, and deeper understanding of naturally occurring facial action.}, keywords = {3D facial expression, Dynamic facial expression database, FACS, Spontaneous expression}, pubstate = {published}, tppubtype = {article} } Facial expression is central to human experience. Its efficiency and valid measurement are challenges that automated facial image analysis seeks to address. Most publically available databases are limited to 2D static images or video of posed facial behavior. Because posed and un-posed (aka “spontaneous”) facial expressions differ along several dimensions including complexity and timing,well-annotated video of un-posed facial behavior is needed. Moreover, because the face is a three-dimensional deformable object, 2D video may be insufficient, and therefore 3D video archives are required.We present a newly developed 3D video database of spontaneous facial expressions in a diverse group of young adults. Well-validated emotion inductions were used to elicit expressions of emotion and paralinguistic communication. Frame-level ground-truth for facial actions was obtained using the Facial Action Coding System. Facial features were tracked in both 2D and 3D domains. To the best of our knowledge, this newdatabase is the first of its kind for the public. Thework promotes the exploration of 3D spatiotemporal features in subtle facial expression, better understanding of the relation between pose and motion dynamics in facial action units, and deeper understanding of naturally occurring facial action. |
2013 |
Zhang, Xing; Yin, Lijun; Cohn, Jeffrey F; Canavan, Shaun; Reale, Michael; Horowitz, Andy; Liu, Peng A high-resolution spontaneous 3d dynamic facial expression database Conference Automatic Face and Gesture Recognition (FG), 2013 10th IEEE International Conference and Workshops on, 2013, ISBN: 978-1-4673-5544-5. Abstract | Links | BibTeX | Tags: 3D facial expression, Dynamic facial expression database, FACS, Spontaneous expression @conference{Zhang2013, title = {A high-resolution spontaneous 3d dynamic facial expression database}, author = {Xing Zhang and Lijun Yin and Jeffrey F Cohn and Shaun Canavan and Michael Reale and Andy Horowitz and Peng Liu}, url = {http://ieeexplore.ieee.org/document/6553788/}, doi = {10.1109/FG.2013.6553788}, isbn = {978-1-4673-5544-5}, year = {2013}, date = {2013-04-22}, booktitle = {Automatic Face and Gesture Recognition (FG), 2013 10th IEEE International Conference and Workshops on}, abstract = {Abstract: Facial expression is central to human experience. Its efficient and valid measurement is a challenge that automated facial image analysis seeks to address. Most publically available databases are limited to 2D static images or video of posed facial behavior. Because posed and un-posed (aka “spontaneous”) facial expressions differ along several dimensions including complexity and timing, well-annotated video of un-posed facial behavior is needed. Moreover, because the face is a three-dimensional deformable object, 2D video may be insufficient, and therefore 3D video archives are needed. We present a newly developed 3D video database of spontaneous facial expressions in a diverse group of young adults. Well-validated emotion inductions were used to elicit expressions of emotion and paralinguistic communication. Frame-level ground-truth for facial actions was obtained using the Facial Action Coding System. Facial features were tracked in both 2D and 3D domains using both person-specific and generic approaches. The work promotes the exploration of 3D spatiotemporal features in subtle facial expression, better understanding of the relation between pose and motion dynamics in facial action units, and deeper understanding of naturally occurring facial action.}, keywords = {3D facial expression, Dynamic facial expression database, FACS, Spontaneous expression}, pubstate = {published}, tppubtype = {conference} } Abstract: Facial expression is central to human experience. Its efficient and valid measurement is a challenge that automated facial image analysis seeks to address. Most publically available databases are limited to 2D static images or video of posed facial behavior. Because posed and un-posed (aka “spontaneous”) facial expressions differ along several dimensions including complexity and timing, well-annotated video of un-posed facial behavior is needed. Moreover, because the face is a three-dimensional deformable object, 2D video may be insufficient, and therefore 3D video archives are needed. We present a newly developed 3D video database of spontaneous facial expressions in a diverse group of young adults. Well-validated emotion inductions were used to elicit expressions of emotion and paralinguistic communication. Frame-level ground-truth for facial actions was obtained using the Facial Action Coding System. Facial features were tracked in both 2D and 3D domains using both person-specific and generic approaches. The work promotes the exploration of 3D spatiotemporal features in subtle facial expression, better understanding of the relation between pose and motion dynamics in facial action units, and deeper understanding of naturally occurring facial action. |
Publication List
2016 |
Multimodal spontaneous emotion corpus for human behavior analysis Conference Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2016, 2016. |
2015 |
Mouth Gesture based Emotion Awareness and Interaction in Virtual Reality Conference SIGGRAPH '15 ACM SIGGRAPH 2015 Posters, (26), SIGGRAPH ACM Digital Library, 2015, ISBN: 978-1-4503-3632-1 . |
2014 |
BP4D-Spontaneous: a high-resolution spontaneous 3D dynamic facial expression database Journal Article Image and Vision Computing, 32 (10), pp. 692-706, 2014. |
2013 |
A high-resolution spontaneous 3d dynamic facial expression database Conference Automatic Face and Gesture Recognition (FG), 2013 10th IEEE International Conference and Workshops on, 2013, ISBN: 978-1-4673-5544-5. |