Quantum-inspires adaptive learning rate optimization (QIALRO)

Abstract

The study proposes an optimization algorithm for machine learning, called Quantum-Inspired Adaptative Learning Rate Optimization (QIALRO), inspired by principles of quantum mechanics. Although the algorithm operates in the classical domain, its conception is based on the ability of quantum systems to simultaneously explore multiple potential states, a property that is sought to be emulated in the optimization of machine learning models. It has been implemented based on softmax regression, cross entropy loss, RMSProp optimizer for learning rate adaptation. The learning rate adjustment mechanism is inspired by the quantum search for optimal solutions by simultaneously exploring multiple possibilities. It adjusts the learning rate by increasing it when the current iteration of the model shows an improvement in loss, leading to an optimal solution space. The technique also permits a decrease in the rate by mimicking the reversion to previous states observed in quantum computing.