基于深度图像和骨骼信息的人体动作识别方法

中国民航大学学报 ›› 2021, Vol. 39 ›› Issue (2): 54-60.

基于深度图像和骨骼信息的人体动作识别方法

张良，钱毅敏

（中国民航大学电子信息与自动化学院，天津 300300）

收稿日期:2020-04-16 修回日期:2020-04-16 接受日期:2020-01-10 出版日期:2021-04-10 发布日期:2021-11-27
作者简介:张良（1970—），男，山东泰安人，教授，博士，研究方向为计算机视觉、机器学习、模式识别.
基金资助:
国家自然科学基金项目（61179045）

Action recognition method based on depth image and skeleton information

ZHANG Liang, QIAN Yimin

College of Electronics and A utomation, CAUC, Tianjin 300300, China

Received:2020-04-16 Revised:2020-04-16 Accepted:2020-01-10 Online:2021-04-10 Published:2021-11-27

摘要/Abstract

摘要：

人体动作的复杂性，类内差异和视角变化等因素，对准确识别人体动作造成一定影响。基于深度相机能同时记录深度图像和实时提取骨骼信息的优势，提出一种基于深度图像和骨骼信息的动作识别方法。使用深度图像序列生成运动历史点云并提取全局特征，同时从三维骨骼信息中提取局部动作特征，对得到的点云特征和骨骼特征进行特征融合，构建出一种多模特征融合的动作识别方法。在 MSR-Action3D 和 UTKinectAction3D 两个数据集进行验证，均达到了95%的识别率，证明了该方法的有效性。

关键词: , 动作识别, 深度图像, 骨骼信息, 特征融合

Abstract: The complexity of human body movements, within-class differences and perspective changes hinder the accurate recognition of human movements. Depth camera can simultaneously record depth images and extract skeleton information; basing on that, a motion recognition method is proposed. Depth image sequence is used to generate a motion history point cloud and extract global features and local motion features from 3D skeleton information. Point cloud features and skeleton features are fused to construct a multi-modal feature fusion action recognition method. Test on two data sets of MSR-Action3D and UTKinect-Action3D shows that recognition rates are above 95% and 98.0%, which proves the effectiveness of the method

Key words: action recognition, depth maps, skeleton information, feature fusion

中图分类号:

TP391.4

张良, 钱毅敏. 基于深度图像和骨骼信息的人体动作识别方法[J]. 中国民航大学学报, 2021, 39(2): 54-60.

ZHANG Liang, QIAN Yimin. Action recognition method based on depth image and skeleton information [J]. Journal of Civil Aviation University of China, 2021, 39(2): 54-60.

参考文献 23

[1]	WEINLAND D, BOYER E援 Action recognition using exemplar-basedembedding[C]//2008 IEEE Conference on Computer Vision and PatternRecognition,June 23-28, 2008, Anchorage, AK, USA援 IEEE, 2008: 1-7援
[2]	GUO K, ISHWAR P, KONRAD J援 Action recognition in video by sparserepresentation on covariance manifolds of silhouette tunnels[C]//Recognizing Patterns in Signals, Speech, Images and Videos(ICPR), Istanbuld, Turkey, August 23-26, 2010: 294-305援
[3]	张良, 鲁梦梦, 姜华援局部分布信息增强的视觉单词描述与动作识别[J]援电子与信息学报, 2016, 38(3): 549-556援
[4]	YANG X D, ZHANG C Y, TIAN Y L援 Recognizing actions using depthmotion maps-based histograms of oriented gradients[C]//Proceedings ofthe 20th ACM International Conference on Multimedia, October 29 -November 2, 2012, Nara, Japan援 New York: ACM Press, 2012: 1057 -1060援
[5]	RAHMANI H, MAHMOOD A, DU Q H, et al援 HOPC: histogram of oriented principal components of 3D pointclouds for action recognition[C]//Computer Vision -ECCV 2014, Zurich, Switzerland, September 6 -12, 2014: 742-757援
[6]	LIU W P, JIANG Y F, WANG H L, et al援 Action description usingpointclouds [C]//Second International Workshop on Pattern Recognition, Singapore, May 1-3, 2017, 1044: 104430X援
[7]	XIA L, CHEN C C, AGGARWAL J K援 View invariant human actionrecognition using histograms of 3D joints[C]//2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, June 16-21, 2012, Providence, RI, USA援 IEEE, 2012: 20-27援
[8]	CARBONERA LUVIZON D, TABIA H, PICARD D援Learning featurescombination for human action recognition from skeleton sequences[J]援Pattern Recognition Letters, 2017, 99: 13-20援
[9]	毛峡, 王岚, 李建军援一种基于 RGB-D 特征融合的人体行为识别框架[J]援计算机科学, 2018, 45(8): 22-27援
[10]	J魪GOU H, DOUZE M, SCHMID C, et al援 Aggregating local descriptorsinto a compact image representation[C]//2010 IEEE Computer SocietyConference on Computer Vision and Pattern Recognition, June 13-18, 2010, San Francisco, CA, USA援 IEEE, 2010: 3304-3311援
[11]	ZEILER M D, FERGUS R援 Visualizing and understanding convolutionalnetworks[C]//Computer Vision-ECCV 2014, Zurich, Switzerland, September6-12, 2014: 818-833援
[12]	李玉援基于 3D 激光点云的无人车城市环境 SLAM 问题研究[D]援北京: 北京理工大学, 2016援
[13]	RUSU R B, BRADSKI G, THIBAUX R, et al援 Fast 3D recognition andpose using the Viewpoint Feature Histogram[C]//2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, October 18-22, 2010, Taipei, Taiwan, China援 IEEE, 2010: 2155-2162援
[14]	ZHANG H L, ZHONG P, HE J L, et al援 Combining depth-skeleton feature with sparse coding for action recognition[J]援 Neurocomputing, 2017, 230: 417-426援
[15]	李佩援基于 Kinect 骨骼数据的人体动作识别方法研究[D]援郑州:郑州大学, 2017援
[16]	CORTES C, VAPNIK V援 Support-vector networks[J]援 Machine Learning, 1995, 20(3): 273-297援
[17]	刘婷婷, 李玉鹏, 张良援多视角深度运动图的人体行为识别[J]援中国图象图形学报, 2019, 24(3): 400-409援
[18]	YANG X D, TIAN Y L援 EigenJoints -based action recognition usingNat觙ve-Bayes-Nearest-Neighbor[C]//2012 IEEE Computer SocietyConference on Computer Vision and Pattern Recognition Workshops, June 16-21, 2012, Providence, RI, USA援 IEEE, 2012: 14-19援
[19]	LI W Q, ZHANG Z Y, LIU Z C援 Action recognition based on a bag of 3Dpoints[C]//2010 IEEE Computer Society Conference on Computer Visionand Pattern Recognition Workshops (CVPRW), June 13-18, 2010, SanFrancisco, CA, USA援 IEEE, 2010: 9-14援
[20]	OREIFEJ O, LIU Z C援 HON4D: histogram of oriented 4D normals foractivity recognition from depth sequences[C]//2013 IEEE Conferenceon Computer Vision and Pattern Recognition, June 23-28, 2013, Portland, OR, USA援 IEEE, 2013: 716-723援
[21]	YANG X D, TIAN Y L援 Super normal vector for human activity recognition with depth cameras[J]援 IEEE Transactions on Pattern AnalysisandMachine Intelligence, 2017, 39(5): 1028-1039援
[22]	VEMULAPALLI R, ARRATE F, CHELLAPPA R援 Human action recognition by representing 3D skeletons as points in a lie group[C]//2014IEEE Conference on Computer Vision and Pattern Recognition, June23-28, 2014, Columbus, OH, USA援 IEEE, 2014: 588-595援
[23]	DEVANNE M, WANNOUS H, BERRETTI S, et al援3-D human actionrecognition by shape analysis of motion trajectories on Riemannian manifold[J]援IEEE Transactions on Cybernetics, 2015, 45(7): 1340-1352援