Interpretable policies for reinforcement learning by empirical fuzzy sets

Huang, J. and Angelov, Plamen P. and Yin, C. (2020) Interpretable policies for reinforcement learning by empirical fuzzy sets. Engineering Applications of Artificial Intelligence, 91: 103559. ISSN 0952-1976

[thumbnail of Binder1]
Text (Binder1)
Binder1.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (2MB)

Abstract

This paper proposes a method and an algorithm to implement interpretable fuzzy reinforcement learning (IFRL). It provides alternative solutions to common problems in RL, like function approximation and continuous action space. The learning process resembles that of human beings by clustering the encountered states, developing experiences for each of the typical cases, and making decisions fuzzily. The learned policy can be expressed as human-intelligible IF-THEN rules, which facilitates further investigation and improvement. It adopts the actor–critic architecture whereas being different from mainstream policy gradient methods. The value function is approximated through the fuzzy system AnYa. The state–action space is discretized into a static grid with nodes. Each node is treated as one prototype and corresponds to one fuzzy rule, with the value of the node being the consequent. Values of consequents are updated using the Sarsa() algorithm. Probability distribution of optimal actions regarding different states is estimated through Empirical Data Analytics (EDA), Autonomous Learning Multi-Model Systems (ALMMo), and Empirical Fuzzy Sets ( ε FS). The fuzzy kernel of IFRL avoids the lack of interpretability in other methods based on neural networks. Simulation results with four problems, namely Mountain Car, Continuous Gridworld, Pendulum Position, and Tank Level Control, are presented as a proof of the proposed concept.

Item Type:
Journal Article
Journal or Publication Title:
Engineering Applications of Artificial Intelligence
Additional Information:
This is the author’s version of a work that was accepted for publication in Engineering Applications of Artificial Intelligence. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Engineering Applications of Artificial Intelligence, 91, 2020 DOI: 10.1016/j.engappai.2020.103559
Uncontrolled Keywords:
/dk/atira/pure/subjectarea/asjc/1700/1702
Subjects:
?? interpretable fuzzy systemsreinforcement learningprobability distribution learningautonomous learning systemsanya type fuzzy systemsempirical fuzzy setsartificial intelligencecontrol and systems engineeringelectrical and electronic engineering ??
ID Code:
142272
Deposited By:
Deposited On:
12 Mar 2020 11:20
Refereed?:
Yes
Published?:
Published
Last Modified:
25 Aug 2024 23:49