۱- Introduction

۲- Theoretical background

۳- Formulation of the problem

۴- Development of RL-based RTS using the MDRs mechanism

۵- Experiment

۶- Conclusion and future work

References

Abstract

Previous studies of the real-time scheduling (RTS) problem domain indicate that using a multiple dispatching rules (MDRs) strategy for the various zones in the system can enhance the production performance to a greater extent than using a single dispatching rule (SDR) over a given scheduling interval for all the machines in the shop floor control system. This approach is feasible but the drawback of the previously proposed MDRs method is its inability to respond to changes in the shop floor environment. The RTS knowledge base (KB) is not static, so it would be useful to establish a procedure that maintains the KB incrementally if important changes occur in the manufacturing system. To address this issue, we propose reinforcement learning (RL)-based RTS using the MDRs mechanism by incorporating two main mechanisms: (1) an off-line learning module and (2) a Q-learning-based RL module. According to various performance criteria over a long period, the proposed approach performs better than the previously proposed MDRs method, the machine learning-based RTS using the SDR approach, and heuristic individual dispatching rules.

Introduction

Industry 4.0, also called “Smart Factory,” aims to increase factory productivity and the efficient utilization of resources in real time (Herrmann, Pentek, & Otto, 2015; Wang, Wan, Li, & Zhang, 2016). These objectives are achieved via flexible event-driven reactions to changes in the factory environment, resource allocation, scheduling, optimization, and control in real time. Most of the “Smart Factory” concepts share the attributes of cyber-physical systems (CPS) for monitoring physical processes by creating a virtual copy of the physical world and making decentralized decisions (Lee, Bagheri, & Kao, 2015). CPS is defined as a transformative technology for managing interconnected systems according to their physical assets and computational capabilities, and recent developments have improved the availability and affordability of sensors, data acquisition systems, and computer networks (Lee, 2008; Wolf, 2009). The competitive nature of current industry is forcing more factories to implement high-tech methods. Thus, the increasing use of sensors, RFID, and networked machines has resulted in the continuous generation of high volume data known as Big Data (Lee, Lapira, Bagheri, & Kao, 2013; Lee, Kao, & Yang, 2014). In this environment, CPS can be developed further to manage Big Data and exploit the interconnectivity among machines to fulfill the goal of producing intelligent, resilient, and self-adaptable machines. Furthermore, by integrating CPS with production, logistics, and services in current industrial practices, it will be possible to transform current factories into Industry 4.0 factories with significant economic potential. This is why it is timely and crucial to consider adaptive scheduling and control (i.e., real-time scheduling; RTS) for dynamic manufacturing environments as key research issues in CPS production management (Goryachev et al., 2013; Kück et al., 2016).