Sensors & Transducers

Vol. 261, Issue 2, July 2023, pp. 25-32

Learning from Demonstration and Safe Cobotics Using
Digital Twins


1 School of Engineering Neuchâtel, University of Applied Sciences and Arts Western Switzerland, HES-SO/HE-ARC, Espace de l’Europe 11, 2000 Neuchâtel, Switzerland

2 School of Engineering and Management Vaud, University of Applied Sciences and Arts Western Switzerland, HES-SO/HEIG-VD, Route de Cheseaux 1,
1401 Yverdon-les-Bains, Switzerland

1 Tel.: +41 32 930 22 94, fax: +41 32 930 22 94


Received: 14 April 2023 /Accepted: 30 May 2023 /Published: 26 June 2023

Abstract: The use of collaborative robots, or cobots, is nowadays continually increasing, especially in the small- and medium-sized manufacturing sector. For each particular use case, the integration and deployment of a cobot into a collaborative workspace faces a certain number of challenges. Programming industrial robots, for example, can be a relatively complex and time-consuming task. In this paper we report an accurate method to robot programming by using an optimized “learning from demonstration” technique. The operator/programmer performs in real-time the corresponding task to be automatized, and by means of a tracker sensor the programmer’s motions are captured and transmitted to the robot the robot registers the trajectories and is now able to reproduce the human movements with high accuracy. Another fundamental issue for cobot deployment is safety. In this paper, we also present a virtual/augmented reality (VR/AR) environment to facilitate the design and operation of cobots in order to maximize human safety. The virtual reality environment operates as an aide tool during the design phase. The human operator and the robot’s digital twin work side-by-side while executing a collaborative task in a virtual reality space. Their movements are controlled and registered, and after a given period of test time, the data is analyzed to suggest modifications to ensure a safe workspace (collision free) and to increase productivity. For the regular real-time cobot operation, an augmented reality environment was developed, again, with the purpose of assuring a safe human-robot collaboration. The augmented reality environment keeps tracking permanently the cobot and the human manipulations. This system produces audio and visual alarm signals in unsafe situations and is also able to take actions, such as slowing down or stopping the robot, to preserve the physical integrity of the human operator.

Keywords: Cobot, Human-robot collaboration, Virtual- and augmented reality, Learning from demonstration, Machine learning, Digital twin, Sensors, Smart glasses.