基于强化学习的模块机器人故障自修复方法

中国民航大学学报 ›› 2023, Vol. 41 ›› Issue (1): 52-57.

基于强化学习的模块机器人故障自修复方法

管恩广^1，2，王尧¹，曹家彬²，赵言正²

（1. 上海海事大学物流工程学院，上海 201306； 2. 上海交通大学机械与动力工程学院，上海 200240）

收稿日期:2022-01-08 修回日期:2022-04-18 出版日期:2023-10-29 发布日期:2023-10-29
作者简介:管恩广（1983—），男，黑龙江哈尔滨人，讲师，博士，研究方向为移动机器人控制理论方法.
基金资助:
国家自然科学基金青年科学基金项目（61806124）

Fault self-repair method of modular robot based on reinforcement learning algorithm

Guan Enguang^1,2 , Wang Yao¹ , Cao Jiabin² , Zhao Yanzheng²

(1. Logistics Engineering College, Shanghai Maritime University, Shanghai 201306, China;2. School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China)

Received:2022-01-08 Revised:2022-04-18 Online:2023-10-29 Published:2023-10-29

摘要/Abstract

摘要： 针对晶格式模块机器人的鲁棒性设计问题，提出一种基于强化学习算法的故障自修复方法。该方法将系统内以空位填充为目标的自修复过程转化为以包含空位的子群模块运动为手段的系统自重构过程。同时，基于强化学习算法提出一种离散方式下的空位移动路径规划，并按此引导空位在系统内穿行。仿真试验结果表明，该故障自修复方法的有效性在晶格式模块机器人系统上得到了验证，且可广泛应用于其他同构模块机器人系统

关键词: 模块机器人, 自重构, 强化学习算法, 故障自修复

Abstract: A fault self-repair method based on reinforcement learning algorithm is proposed for the robust design issue of lattice modular robot. The method transforms the self-repair process with the goal of filling empty spaces into a self-reconfiguration process by means of the movements of meta-modules containing empty nodes. Based on the reinforcement learning algorithm, a discrete path planning for the movement of empty nodes is proposed, and the empty nodes are accordingly guided to travel through the system. The simulation test results show that the effectiveness of this fault self-repair method is verified on the lattice modular robotic system and can be widely applied to other isomorphic modular robotic systems

Key words: modular robot, self-reconfiguration, reinforcement learning algorithm, fault self-repair

中图分类号:

TH113.22

管恩广, 王尧, 曹家彬, 赵言正. 基于强化学习的模块机器人故障自修复方法[J]. 中国民航大学学报, 2023, 41(1): 52-57.

Guan Enguang , Wang Yao , Cao Jiabin, Zhao Yanzheng. Fault self-repair method of modular robot based on reinforcement learning algorithm[J]. Journal of Civil Aviation University of China, 2023, 41(1): 52-57.

参考文献 13

[1]	SPROEWITZ A, LAPRADE P, BONARDI S, et al Roombots—towards decentralized reconfiguration with self -reconfiguring modular robotic metamodules[C]//2010 IEEE/RSJ International Conference on Intelli－ gent Robots and Systems, October 18-22, 2010, Taibei, China. IEEE, 2010: 1126-1132.
[2]	TAN W S, WEI H X, YANG B. SambotII: a new self-assembly modular robot platform based on sambot[J]. Applied Sciences, 2018, 8(10): 17191726.
[3]	SALEMI B, MOLL M, SHEN W M. SUPERBOT: a deployable, multifunctional, and modular self -reconfigurable robotic system [C]//2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, October 9-15, 2006, Beijing, China. IEEE, 2006: 3636-3641.
[4]	PARROTT C, DODD T J, GRO R. HyMod: a 3-DOF hybrid mobile and self-reconfigurable modular robot and its extensions[M]//Distribut－ ed Autonomous Robotic Systems. Cham: Springer, 2018: 401-414.
[5]	CHRISTENSEN D J. Experiments on fault-tolerant self-reconfiguration and emergent self-repair[C]//2007 IEEE Symposium on Artificial Life.
	IEEE, 2007: 355-361.
[6]	GILPIN K, KOTAY K, RUS D, et al Miche: modular shape formation by self-disassembly[J]. The International Journal of Robotics Research, 2008, 27(3/4): 345-372.
[7]	ZHU J W, CHEN M Z Q, SU S H. Distributed algorithms for shape sculpting of lattice -arrayed modular robots via hole motion [C]//2013 10th IEEE International Conference on Control andAutomation(ICCA), June 12-14, 2013, Hangzhou, China. IEEE, 2013: 135-140.
[8]	DUGAS D, NIETO J, SIEGWART R, et al NavRep: unsupervised representations for reinforcement learning of robot navigation in dynamic human environments[C]//2021 IEEE International Conference on Robotics and Automation(ICRA). New York: ACM, 2021: 7829-7835.
[9]	CHRISTENSEN D J, SCHULTZ U P, STOY K. A distributed and mor－ phology-independent strategy for adaptive locomotion in self-reconfig－ urable modular robots[J]. Robotics and Autonomous Systems, 2013, 61(9): 1021-1035.
[10]	GUAN E, LIU J, ZHAO Y. Self-configuration strategy design for unitcompressible modular robotic system[C]//CSAA/IET International Con－ ference on Aircraft Utility Systems, September 18-21, 2020. London: IET, 2021: 232-237.
[11]	DE ROSA M, GOLDSTEIN S, LEE P, et al Scalable shape sculpting via hole motion: motion planning in lattice-constrained modular robots[C]// IEEE International Conference on Robotics and Automation, May 1519, 2006, Orlando, FL, USA. IEEE, 2006: 1462-1468.
[12]	WATKINS C J C H, DAYAN P. Q -learning [J]. Machine Learning, 1992, 8(3/4): 279-292.
[13]	KOBER J, BAGNELL J A, PETERS J. Reinforcement learning in robotics: a survey[J]. The International Journal of Robotics Research, 2013, 32(11): 1238-1274.