An Intelligent Framework for Autonomous Robot-based Machine-tending Applications

• Author / Creator

  Jia, Feiyu

• The manufacturing sector stands as a fundamental pillar of worldwide economies, contributing markedly to global economic expansion. Over the past five years, the implementation of automated machine-tending systems has widely extended from simulation or laboratory environments to real-world scenarios in manufacturing workshops, as robotics and artificial intelligence develop rapidly. Machine tending is a critical part of the manufacturing process through interaction with the machine and surrounding environments. Currently, most of the machine tending tasks are still carried out manually or via collaborative robots by cooperating with humans on site. However, with the development of AI-enabled robots, intelligent manufacturing has been moving from mass production to mass customization and uses robots and artificial intelligence techniques to minimize human interventions in manufacturing activities. Inspection of the machine’ working status is critical in manufacturing processes, ensuring that machines work correctly without any interruptions, e.g., in lights-out manufacturing. In addition, autonomous robot-based machine tending applications are necessary for smart factories with the increasing demand for full autonomy. So far, there is no attempt has been made toward the framework for fully autonomous robot-based machine-tending applications. Consequently, this research aims to develop an intelligent framework to attend CNC machines by integrating autonomous mobile manipulation systems, scene text recognition, real-time object detection, position estimation and path planning techniques to achieve fully autonomous operations of the mobile manipulator in manufacturing environments, which consist of four main stages: 1) a path planning, and docking to charging station method for autonomous mobile manipulator system to enable system move between different workstations and autonomous charging for continuously and smoothly working; 2) an automatic object detection method for the machine-tending system to identify the target machine in the workspace; 3) an intelligent inspection of machine’s working status through command recognition; 4) a button detection and localization method to support the manipulation by moving toward the control buttons to execute machine instructions.
  In the first stage, an autonomous mobile manipulation system (AMM) is proposed for part handling, loading and unloading, and some other auxiliary tasks in machine-tending. In addition, an improved path-planning algorithm based on Rapidly-exploring Random Tree (RRT) and the quintic B-spline curve technique is proposed for robotic machine-tending systems to move between the workstation and the charging dock. Furthermore, an autonomous docking and charging method is developed for machine-tending systems to work continuously in manufacturing environments. This method requires two steps: i) detecting the charging station, typically in an unstructured environment, and ii) autonomously docking to the charging station. For charging station detection, a YOLOv7-based method is developed to quickly and accurately recognize the charging station.
  The second stage of the proposed framework is the identification of the target CNC machine. It is important to note that there is often more than one CNC machine working in a representative manufacturing environment. Therefore, a deep learning-based machine detection method, called SiameseRPN, is developed to recognize the specific machine from a group of machines in the workspace. This method combines the Siamese neural network and region proposal network.
  In the third part, a command recognition method by integrating the text region proposal network, recurrent neural network and connectionist temporal classification was proposed to read and understand the CNC machine instruction for further operation. To improve the accuracy of recognition performance, a dictionary-guided procedure is also proposed.
  In the last stage, a benchmark dataset for five different types of control buttons on the Haas CNC machine is created and the YOLOv7-based benchmark button detection method is developed to identify and localize the target buttons recognized in the machine commands to assist in the instructions executions of robotic machine-tending systems.

• Subjects / Keywords

  • Manufacturing
  • Machine Tending

• Graduation date

  Spring 2024

• Type of Item

  Thesis

• Degree

  Doctor of Philosophy

• DOI

  https://doi.org/10.7939/r3-4rwa-6849

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.