Optimum Operation and Control of a Batch Chemical Process using Reinforcement Learning

Abstract

A Reinforcement learning (RL) approach is presented as a new automatic learning approach to the
problem of optimal operation and control of batch chemical processes (e.g. batch reactors and batch distillation
columns). The approach is particularly suited to batch process optimization problems, especially through not
assuming prior detailed process knowledge or availability of a process model. The particular suitability of RL as a
framework for optimising batch process operation has been recognised already (Martinez et al. (1998a,b)). In the
implementation of RL, use is made of how the plant responds to control actions aggregated in generalised
‘predictive’ models, each linking adjacent intra-batch decision steps. The relative worth of a control action at a
decision step is aggregated in the ‘value function’. Both the predictive models and the value function are learned
from the accumulating measurement data, batch-to-batch, starting from a small initial set of test batches. The
methodology is exemplified using two simple batch process case studies which were used to test the MATLAB
computer coding of the RL algorithm. In addition, issues regarding the structure of the initial training data set and
the embedded Neural Network have been investigated.