On model-free and model-based techniques for the quadratic control of Markov jump linear systems with unknown transition probabilities

Rafael L. Beirigo¹; Marcos G. Todorov¹; André da Motta Salles Barreto²

¹ Laboratório Nacional de Computação Científica - LNCC; ² Laboratório Nacional de Computação Científica

Baixar PDF doi:10.20906/CPS/CBA2018-0512

Resumo

Markov jump linear systems encompass a theoretically sound and solid framework for pursuing the optimal control of systems with switching dynamics. Despite its broad applicability, demand for a priori perfect knowledge of the transition model may render the application of the solution techniques impractical. To circumvent this limitation, two techniques were recently proposed that prescind from the perfect prior knowledge of the transition model. In the face of one being model-free, and the other model-based the promising results that were presented by each technique separately may admit a comparative analysis. Here, we provide an experimental evaluation of both techniques, applying them to the control in a simulator of a robotic arm whose joints are subject to failure. Additionally, we test two variations of the policy update strategy for the model-free technique. The experimental results suggest a comparable performance for both techniques.

Palavras-chave: Markov jump linear systems; reinforcement learning; adaptive control; robotics