Simulation of a simple E-Robot Warehouse Environment using Reinforcement Learning

Abstract

Warehousing is an important part of a supply chain network. There are several types of operations that can occur in a warehouse environment; where most of the costs are associated with labor, mainly related to order-picking in warehouse building(s). Automating certain processes in a warehouse is a way to try to reduce labor costs. There are different types of robots that can be deployed for such purpose, depending on the task(s) they are to carry out. One kind of growing interest are Autonomous Mobile Robots (AMRs), which use hardware and software to navigate their surroundings without requiring a pre-planned routing to perform their work. Efficiently coordinating a fleet of AMRs, with a specified set of rules and goals to follow, represents a complex optimization challenge, being an active topic of research and development, due to the positive impact this has and could have, such as the reduction of costs linked to processes involved in warehousing operations. The objective of this project was to develop a reinforcement learning model environment that simulates, to certain extend, the conditions and rules involved in a warehouse, to perform the order-picking process from storage/picking locations to delivery/shipping locations, by a fleet of AMRs powered by electric batteries; using as fewer computational resources as possible, so that it can run on a personal computer, with moderate specs. Six different experiments cases were carried out, whose results demonstrate the development of a simple RL model, controlling several robots (AMRs), capable of handling the “movement” of “packages” from “storing/picking locations” to “shipping/delivery locations” and the “recharging” of their “batteries”; with a decrease in performance, with the increase in the complexity of the environment and the number of robots; within the constraints and considerations of the RL model development.