Documentation
This stack provides a ROS based framework for performing reinforcement learning (RL) and packages of RL agents and environments.
The code in this repository provides agents, environments, and multiple ways for them to communicate (through ROS messages, or by including the agent and environment libraries). There are 5 packages in the repository:
rl_common: Some files that are common to both agents and environments.
rl_msgs: Definitions of ROS messages for agents and envs to communicate.
rl_agent: A library of some RL agents including Q-Learning and TEXPLORE.
rl_env: A library of some RL environments such as Taxi and Fuel World.
rl_experiment: Code to run some RL experiments without ROS message passing.
More details for each of the packages are available at the above links. References for some of the algorithms and environments provided in this package are available at the author's website.
Checking out the code
Check out the code at: https://github.com/toddhester/rl-texplore-ros-pkg
Tutorial
For details on how to use this stack, check out the tutorial.
Features
Some of the key features of this package include:
Implementations of many RL algorithms (Q-Learning (Watkins 1989), SARSA (Rummery and Niranjan 1994), Dyna (Sutton 1990), TEXPLORE (Hester and Stone 2010), R-Max (Brafman and Tennenholtz 2001)).
Implementations of many RL environments (Gridworlds, Fuel World (Hester and Stone 2010), Taxi (Dietterich 1998), Stock Trading (Strehl et al 2007), Light World (Konidaris and Barto 2007), Mountain Car (Sutton and Barto 1998), Cart Pole (Sutton and Barto 1998), Robot Car Simulation (Beeson et al 2008, Hester et al 2012)).
Uses a set of ROS messages for communicating between RL agents and environment (similar to RL-Glue (Tanner and White 2009)).
Includes the TEXPLORE RL algorithm (Hester and Stone 2010).
Includes our real-time parallel architecture for model-based RL (Hester et al 2012).
- Includes a general model-based agent that can be used with any model learning or planning method.
Agents and environments can be written in any language as long as they use rl_msgs to communicate.
Provides code (rl_experiment package) to run RL experiments and collect reward data.
- Can easily be integrated with robots running ROS to perform learning on them.
- New model learning and planning methods can easily be added and used within the general model based agent and the real time architecture.