Skip to content

Releases: samindaa/RLLib

C++ Template Library to Predict, Control, Learn Behaviors, and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning

13 Apr 00:42
@samindaa samindaa
v2.2
5f6aedf
Compare
Choose a tag to compare
C++ Template Library to Predict, Control, Learn Behaviors, and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning Latest
Latest

RLLib

(C++ Template Library to Predict, Control, Learn Behaviors, and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning)

RLLib is a lightweight C++ template library that implements incremental, standard, and gradient temporal-difference learning algorithms in Reinforcement Learning. It is an optimized library for robotic applications and embedded devices that operates under fast duty cycles (e.g., < 30 ms). RLLib has been tested and evaluated on RoboCup 3D soccer simulation agents, physical NAO V4 humanoid robots, and Tiva C series launchpad microcontrollers to predict, control, learn behaviors, and represent learnable knowledge. The implementation of the RLLib library is inspired by the RLPark API, which is a library of temporal-difference learning algorithms written in Java.

Features

  • Off-policy prediction algorithms:
    • GTD(lambda)
    • GQ(lambda)
  • Off-policy control algorithms:
    • Q(lambda)
    • Greedy-GQ(lambda)
    • Softmax-GQ(lambda)
    • Off-PAC (can be used in on-policy setting)
  • On-policy algorithms:
    • TD(lambda)
    • TD(lambda)AlphaBound
    • TD(lambda)True
    • Sarsa(lambda)
    • Sarsa(lambda)AlphaBound
    • Sarsa(lambda)True
    • Sarsa(lambda)Expected
    • Actor-Critic (continuous actions, discrete actions, discounted reward settting, averaged reward settings, and so on)
  • Supervised learning algorithms:
    • Adaline
    • IDBD
    • KI
    • SemiLinearIDBD
    • Autostep
  • Policies:
    Random
    RandomX%Bias
    Greedy
    Epsilon-greedy
    Boltzmann
    Normal
    Softmax
  • Dot product:
    An efficient implementation of the dot product for tile coding based feature representations (with culling traces).
  • Benchmarking environments:
    Mountain Car
    Mountain Car 3D
    Swinging Pendulum
    Continuous Grid World
    Bicycle
    Cart Pole
    Acrobot
    Non-Markov Pole Balancing
    Helicopter
  • Optimization:
    Optimized for very fast duty cycles (e.g., with culling traces, RLLib has been tested on the Robocup 3D simulator agent, and on the NAO V4 (cognition thread)).
  • Usage:
    The algorithm usage is very much similar to RLPark, therefore, swift learning curve.
  • Examples:
    There are a plethora of examples demonstrating on-policy and off-policy control experiments.
  • Visualization:
    We provide a Qt4 based application to visualize benchmark problems.

Extension

Extension for Tiva C Series EK-TM4C123GXL LaunchPad, and Tiva C Series TM4C129 Connected LaunchPad microcontrollers.

Tiva C series launchpad microcontrollers: https://github.com/samindaa/csc688

Demo

Off-PAC ContinuousGridworld
AverageRewardActorCritic SwingPendulum (Continuous Actions)

Usage

RLLib is a C++ template library. The header files are located in the include directly. You can simply include/add this directory from your projects, e.g., -I./include, to access the algorithms.

To access the control algorithms:

#include "ControlAlgorithm.h"

To access the predication algorithms:

#include "PredictorAlgorithm"

To access the supervised learning algorithms:

#include "SupervisedAlgorithm.h"

RLLib uses the namespace:

using namespace RLLib

Testing

RLLib provides a flexible testing framework. Follow these steps to quickly write a test case.

  • To access the testing framework: #include "HeaderTest.h"
#include "HeaderTest.h"

RLLIB_TEST(YourTest)

class YourTest Test: public YourTestBase
{
  public:
    YourTestTest() {}

    virtual ~Test() {}
    void run();

  private:
    void testYourMethod();
};

void YourTestBase::testYourMethod() {/** Your test code */}

void YourTestBase::run() { testYourMethod(); }
  • Add YourTest to the test/test.cfg file.
  • You can use @YourTest to execute only YourTest. For example, if you need to execute only MountainCar test cases, use @MountainCarTest.

Test Configuration

The test cases are executed using:

  • ./configure
  • make
  • ./RLLib
  • To compile 32-bit binary on a 64-bit machine:
    • ./configure --march=m32
    • make
    • ./RLLib
  • Debugging:
    • ./configure --debug=ggdb
    • make
    • ./RLLib

Visualization

RLLib provides a QT5 based Reinforcement Learning problems and algorithms visualization tool named RLLibViz. Currently RLLibViz visualizes following problems and algorithms:

  • On-policy:
    • SwingPendulum problem with continuous actions. We use AverageRewardActorCritic algorithm.
  • Off-policy:
    • ContinuousGridworld and MountainCar problems with discrete actions. We use Off-PAC algorithm.
  • In order to run the visualization tool, you need to have QT4.8 installed in your system.
  • In order to install RLLibViz:
    • Change directory to visualization/RLLibViz
    • qmake RLLibViz.pro
    • make
    • ./RLLibViz

Documentation

Operating Systems

  • Ubuntu >= 11.04
  • Windows (Visual Studio 2013)
  • Mac OS

Publications

Contact

Saminda Abeyruwan (saminda@cs.miami.edu)

Assets 2

C++ Template Library to Predict, Control, Learn Behaviors, and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning

14 Oct 20:21
@samindaa samindaa
v2.1
a4a433a
Compare
Choose a tag to compare
C++ Template Library to Predict, Control, Learn Behaviors, and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning

RLLib

(C++ Template Library to Predict, Control, Learn Behaviors, and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning)

RLLib is a lightweight C++ template library that implements incremental, standard, and gradient temporal-difference learning algorithms in Reinforcement Learning. It is an optimized library for robotic applications and embedded devices that operates under fast duty cycles (e.g., < 30 ms). RLLib has been tested and evaluated on RoboCup 3D soccer simulation agents, physical NAO V4 humanoid robots, and Tiva C series launchpad microcontrollers to predict, control, learn behaviors, and represent learnable knowledge. The implementation of the RLLib library is inspired by the RLPark API, which is a library of temporal-difference learning algorithms written in Java.

Features

  • Off-policy prediction algorithms:
    • GTD(lambda)
    • GQ(lambda)
  • Off-policy control algorithms:
    • Greedy-GQ(lambda)
    • Softmax-GQ(lambda)
    • Off-PAC (can be used in on-policy setting)
  • On-policy algorithms:
    • TD(lambda)
    • TD(lambda)AlphaBound
    • TD(lambda)True
    • Sarsa(lambda)
    • Sarsa(lambda)AlphaBound
    • Sarsa(lambda)True
    • Sarsa(lambda)Expected
    • Actor-Critic (continuous actions, discrete actions, discounted reward settting, averaged reward settings, and so on)
  • Supervised learning algorithms:
    • Adaline
    • IDBD
    • KI
    • SemiLinearIDBD
    • Autostep
  • Policies:
    Random
    RandomX%Bias
    Greedy
    Epsilon-greedy
    Boltzmann
    Normal
    Softmax
  • Dot product:
    An efficient implementation of the dot product for tile coding based feature representations (with culling traces).
  • Benchmarking environments:
    Mountain Car
    Mountain Car 3D
    Swinging Pendulum
    Continuous Grid World
    Bicycle
    Cart Pole
    Acrobot
    Non-Markov Pole Balancing
    Helicopter
  • Optimization:
    Optimized for very fast duty cycles (e.g., with culling traces, RLLib has been tested on the Robocup 3D simulator agent, and on the NAO V4 (cognition thread)).
  • Usage:
    The algorithm usage is very much similar to RLPark, therefore, swift learning curve.
  • Examples:
    There are a plethora of examples demonstrating on-policy and off-policy control experiments.
  • Visualization:
    We provide a Qt4 based application to visualize benchmark problems.

Extension

Extension for Tiva C Series EK-TM4C123GXL LaunchPad, and Tiva C Series TM4C129 Connected LaunchPad microcontrollers.

Tiva C series launchpad microcontrollers: https://github.com/samindaa/csc688

Demo

Off-PAC ContinuousGridworld
AverageRewardActorCritic SwingPendulum (Continuous Actions)

Usage

RLLib is a C++ template library. The header files are located in the src directly. You can simply include this directory from your projects, e.g., -I./src, to access the algorithms.

To access the control algorithms:

#include "ControlAlgorithm.h"

To access the predication algorithms:

#include "PredictorAlgorithm"

To access the supervised learning algorithms:

#include "SupervisedAlgorithm.h"

RLLib uses the namespace:

using namespace RLLib

Testing

RLLib provides a flexible testing framework. Follow these steps to quickly write a test case.

  • To access the testing framework: #include "HeaderTest.h"
#include "HeaderTest.h"

RLLIB_TEST(YourTest)

class YourTest Test: public YourTestBase
{
  public:
    YourTestTest() {}

    virtual ~Test() {}
    void run();

  private:
    void testYourMethod();
};

void YourTestBase::testYourMethod() {/** Your test code */}

void YourTestBase::run() { testYourMethod(); }
  • Add YourTest to the test/test.cfg file.
  • You can use @YourTest to execute only YourTest. For example, if you need to execute only MountainCar test cases, use @MountainCarTest.

Test Configuration

The test cases are executed using:

  • ./configure
  • make
  • ./RLLib
  • To compile 32-bit binary on a 64-bit machine:
    • ./configure --march=m32
    • make
    • ./RLLib
  • Debugging:
    • ./configure --debug=ggdb
    • make
    • ./RLLib

Visualization

RLLib provides a QT4.8 based Reinforcement Learning problems and algorithms visualization tool named RLLibViz. Currently RLLibViz visualizes following problems and algorithms:

  • On-policy:
    • SwingPendulum problem with continuous actions. We use AverageRewardActorCritic algorithm.
  • Off-policy:
    • ContinuousGridworld and MountainCar problems with discrete actions. We use Off-PAC algorithm.
  • In order to run the visualization tool, you need to have QT4.8 installed in your system.
  • In order to install RLLibViz:
    • Change directory to visualization/RLLibViz
    • ./configure
    • ./RLLibVizSwingPendulum
    • ./RLLibVizContinuousGridworld
    • ./RLLibVizMountainCar

Documentation

Publication

Dynamic Role Assignment using General ValueFunctions

Contact

Saminda Abeyruwan (saminda@cs.miami.edu)

Assets 2

RLLib v2.0

25 Apr 00:54
@samindaa samindaa
v2.0
3acb81f
Compare
Choose a tag to compare
RLLib v2.0

C++ Template Library to Predict, Control, Learn Behaviors, and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning

RLLib is a lightweight C++ template library that implements incremental, standard, and gradient temporal-difference learning algorithms in Reinforcement Learning. It is an optimized library for robotic applications and embedded devices that operates under fast duty cycles (e.g., < 30 ms). RLLib has been tested and evaluated on RoboCup 3D soccer simulation agents, physical NAO V4 humanoid robots, and Tiva C series launchpad microcontrollers to predict, control, learn behaviors, and represent learnable knowledge. The implementation of the RLLib library is inspired by the RLPark API, which is a library of temporal-difference learning algorithms written in Java.

Features

  • Off-policy prediction algorithms:
    • GTD(lambda)
    • GQ(lambda)
  • Off-policy control algorithms:
    • Greedy-GQ(lambda)
    • Softmax-GQ(lambda)
    • Off-PAC (can be used in on-policy setting)
  • On-policy algorithms:
    • TD(lambda)
    • TD(lambda)AlphaBound
    • TD(lambda)True
    • Sarsa(lambda)
    • Sarsa(lambda)AlphaBound
    • Sarsa(lambda)True
    • Sarsa(lambda)Expected
    • Actor-Critic (continuous actions, discrete actions, discounted reward settting, averaged reward settings, and so on)
  • Supervised learning algorithms:
    • Adaline
    • IDBD
    • KI
    • SemiLinearIDBD
    • Autostep
  • Policies:
    Random
    RandomX%Bias
    Greedy
    Epsilon-greedy
    Boltzmann
    Normal
    Softmax
  • Dot product:
    An efficient implementation of the dot product for tile coding based feature representations (with culling traces).
  • Benchmarking environments:
    Mountain Car
    Mountain Car 3D
    Swinging Pendulum
    Continuous Grid World
    Bicycle
    Cart Pole
    Acrobot
    Non-Markov Pole Balancing
    Helicopter
  • Optimization:
    Optimized for very fast duty cycles (e.g., with culling traces, RLLib has been tested on the Robocup 3D simulator agent, and on the NAO V4 (cognition thread)).
  • Usage:
    The algorithm usage is very much similar to RLPark, therefore, swift learning curve.
  • Examples:
    There are a plethora of examples demonstrating on-policy and off-policy control experiments.
  • Visualization:
    We provide a Qt4 based application to visualize benchmark problems.

Extension

Extension for Tiva C Series EK-TM4C123GXL LaunchPad, and Tiva C Series TM4C129 Connected LaunchPad microcontrollers.

Tiva C series launchpad microcontrollers: https://github.com/samindaa/csc688

Demo

Off-PAC ContinuousGridworld
AverageRewardActorCritic SwingPendulum (Continuous Actions)

Usage

RLLib is a C++ template library. The header files are located in the src directly. You can simply include this directory from your projects, e.g., -I./src, to access the algorithms.

To access the control algorithms:

#include "ControlAlgorithm.h"

To access the predication algorithms:

#include "PredictorAlgorithm"

To access the supervised learning algorithms:

#include "SupervisedAlgorithm.h"

RLLib uses the namespace:

using namespace RLLib

Testing

RLLib provides a flexible testing framework. Follow these steps to quickly write a test case.

  • To access the testing framework: #include "HeaderTest.h"
#include "HeaderTest.h"

RLLIB_TEST(YourTest)

class YourTest Test: public YourTestBase
{
  public:
    YourTestTest() {}

    virtual ~Test() {}
    void run();

  private:
    void testYourMethod();
};

void YourTestBase::testYourMethod() {/** Your test code */}

void YourTestBase::run() { testYourMethod(); }
  • Add YourTest to the test/test.cfg file.
  • You can use @YourTest to execute only YourTest. For example, if you need to execute only MountainCar test cases, use @MountainCarTest.

Test Configuration

The test cases are executed using:

  • 64-bit machines:
    • ./configure_m64
    • make
    • ./RLLibTest
  • 32-bit machines:
    • ./configure_m32
    • make
    • ./RLLibTest
  • Debugging:
    • ./configure_debug
    • make
    • ./RLLibTest

Visualization

RLLib provides a QT4.8 based Reinforcement Learning problems and algorithms visualization tool named RLLibViz. Currently RLLibViz visualizes following problems and algorithms:

  • On-policy:
    • SwingPendulum problem with continuous actions. We use AverageRewardActorCritic algorithm.
  • Off-policy:
    • ContinuousGridworld and MountainCar problems with discrete actions. We use Off-PAC algorithm.
  • In order to run the visualization tool, you need to have QT4.8 installed in your system.
  • In order to install RLLibViz:
    • Change directory to visualization/RLLibViz
    • ./configure
    • ./RLLibVizSwingPendulum
    • ./RLLibVizContinuousGridworld
    • ./RLLibVizMountainCar

Documentation

Publication

Dynamic Role Assignment using General ValueFunctions

Contact

Saminda Abeyruwan (saminda@cs.miami.edu)

Assets 2

RLLib

01 Dec 09:18
@samindaa samindaa
v1.6
13592c4
Compare
Choose a tag to compare
RLLib

C++ Template Library to Learn Behaviors and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning

RLLib is a lightweight C++ template library that implements incremental, standard, and gradient temporal-difference learning algorithms in Reinforcement Learning. It is a highly optimized library that is designed and written specifically for robotic applications. The implementation of the RLLib library is inspired by the RLPark API, which is a library of temporal-difference learning algorithms written in Java.

Features

  • Off-policy prediction algorithms:
    • GTD(lambda)
    • GQ(lambda)
  • Off-policy control algorithms:
    • Greedy-GQ(lambda)
    • Softmax-GQ(lambda)
    • Off-PAC (can be used in on-policy setting)
  • On-policy algorithms:
    • TD(lambda)
    • TD(lambda)AlphaBound
    • TD(lambda)True
    • Sarsa(lambda)
    • Sarsa(lambda)AlphaBound
    • Sarsa(lambda)True
    • Sarsa(lambda)Expected
    • Actor-Critic (continuous actions, discrete actions, discounted reward settting, averaged reward settings, and so on)
  • Supervised learning algorithms:
    • Adaline
    • IDBD
    • KI
    • SemiLinearIDBD
    • Autostep
  • Policies:
    Random
    RandomX%Bias
    Greedy
    Epsilon-greedy
    Boltzmann
    Normal
    Softmax
  • Dot product:
    An efficient implementation of the dot product for tile coding based feature representations (with culling traces).
  • Benchmarking environments:
    Mountain Car
    Mountain Car 3D
    Swinging Pendulum
    Continuous Grid World
    Bicycle
    Cart Pole
    Acrobot
    Non-Markov Pole Balancing
    Helicopter
  • Optimization:
    Optimized for very fast duty cycles (e.g., with culling traces, RLLib has been tested on the Robocup 3D simulator agent, and on the NAO V4 (cognition thread)).
  • Usage:
    The algorithm usage is very much similar to RLPark, therefore, swift learning curve.
  • Examples:
    There are a plethora of examples demonstrating on-policy and off-policy control experiments.
  • Visualization:
    We provide a Qt4 based application to visualize benchmark problems.

Demo

Off-PAC ContinuousGridworld
AverageRewardActorCritic SwingPendulum (Continuous Actions)

Usage

RLLib is a C++ template library. The header files are located in the src directly. You can simply include this directory from your projects, e.g., -I./src, to access the algorithms.

To access the control algorithms:

#include "ControlAlgorithm.h"

To access the predication algorithms:

#include "PredictorAlgorithm"

To access the supervised learning algorithms:

#include "SupervisedAlgorithm.h"

RLLib uses the namespace:

using namespace RLLib

Testing

RLLib provides a flexible testing framework. Follow these steps to quickly write a test case.

  • To access the testing framework: #include "HeaderTest.h"
#include "HeaderTest.h"

RLLIB_TEST(YourTest)

class YourTest Test: public YourTestBase
{
  public:
    YourTestTest() {}

    virtual ~Test() {}
    void run();

  private:
    void testYourMethod();
};

void YourTestBase::testYourMethod() {/** Your test code */}

void YourTestBase::run() { testYourMethod(); }
  • Add YourTest to the test/test.cfg file.
  • You can use @YourTest to execute only YourTest. For example, if you need to execute only MountainCar test cases, use @MountainCarTest.

Test Configuration

The test cases are executed using:

  • 64-bit machines:
    • ./configure_m64
    • make
    • ./RLLibTest
  • 32-bit machines:
    • ./configure_m32
    • make
    • ./RLLibTest
  • Debugging:
    • ./configure_debug
    • make
    • ./RLLibTest

Visualization

RLLib provides a QT4.8 based Reinforcement Learning problems and algorithms visualization tool named RLLibViz. Currently RLLibViz visualizes following problems and algorithms:

  • On-policy:
    • SwingPendulum problem with continuous actions. We use AverageRewardActorCritic algorithm.
  • Off-policy:
    • ContinuousGridworld and MountainCar problems with discrete actions. We use Off-PAC algorithm.
  • In order to run the visualization tool, you need to have QT4.8 installed in your system.
  • In order to install RLLibViz:
    • Change directory to visualization/RLLibViz
    • ./configure
    • ./RLLibVizSwingPendulum
    • ./RLLibVizContinuousGridworld
    • ./RLLibVizMountainCar

Documentation

Publication

Dynamic Role Assignment using General ValueFunctions

Contact

Saminda Abeyruwan (saminda@cs.miami.edu)

Assets 2

C++ Template Library to Learn Behaviors and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning

20 Nov 00:41
@samindaa samindaa
v1.5
c6ab425
Compare
Choose a tag to compare
C++ Template Library to Learn Behaviors and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning

RLLib

(C++ Template Library to Learn Behaviors and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning)

RLLib is a lightweight C++ template library that implements incremental, standard, and gradient temporal-difference learning algorithms in Reinforcement Learning. It is a highly optimized library that is designed and written specifically for robotic applications. The implementation of the RLLib library is inspired by the RLPark API, which is a library of temporal-difference learning algorithms written in Java.

Features

  • Off-policy prediction algorithms:
    GTD(lambda)
    GQ(lambda)
  • Off-policy control algorithms:
    Greedy-GQ(lambda)
    Softmax-GQ(lambda)
    Off-PAC (can be used in on-policy setting)
  • On-policy algorithms:
    TD(lambda)
    SARSA(lambda)
    Expected-SARSA(lambda)
    Actor-Critic (continuous and discrete actions, discounted, averaged reward settings, etc.)
  • Supervised learning algorithms:
    Adaline
    IDBD
    KI
    SemiLinearIDBD
    Autostep
  • Policies:
    Random
    RandomX%Bias
    Greedy
    Epsilon-greedy
    Boltzmann
    Normal
    Softmax
  • Dot product:
    An efficient implementation of the dot product for tile coding based feature representations (with culling traces).
  • Benchmarking environments:
    Mountain Car
    Mountain Car 3D
    Swinging Pendulum
    Helicopter
    Continuous Grid World
  • Optimization:
    Optimized for very fast duty cycles (e.g., with culling traces, RLLib has been tested on the Robocup 3D simulator agent, and on the NAO V4 (cognition thread)).
  • Usage:
    The algorithm usage is very much similar to RLPark, therefore, swift learning curve.
  • Examples:
    There are a plethora of examples demonstrating on-policy and off-policy control experiments.
  • Visualization:
    We provide a Qt4 based application to visualize benchmark problems.

Demo

Off-PAC ContinuousGridworld
AverageRewardActorCritic SwingPendulum (Continuous Actions)

Usage

RLLib is a C++ template library. The header files are located in the src directly. You can simply include this directory from your projects, e.g., -I./src, to access the algorithms.

To access the control algorithms:

#include "ControlAlgorithm.h"

To access the predication algorithms:

#include "PredictorAlgorithm"

To access the supervised learning algorithms:

#include "SupervisedAlgorithm.h"

RLLib uses the namespace:

using namespace RLLib

Testing

RLLib provides a flexible testing framework. Follow these steps to quickly write a test case.

  • To access the testing framework: #include "HeaderTest.h"
#include "HeaderTest.h"

RLLIB_TEST(YourTest)

class YourTest Test: public YourTestBase
{
  public:
    YourTestTest() {}

    virtual ~Test() {}
    void run();

  private:
    void testYourMethod();
};

void YourTestBase::testYourMethod() {/** Your test code */}

void YourTestBase::run() { testYourMethod(); }
  • Add YourTest to the test/test.cfg file.
  • You can use @YourTest to execute only YourTest. For example, if you need to execute only MountainCar test cases, use @MountainCarTest.

Test Configuration

The test cases are executed using:

  • 64-bit machines:
    • ./configure_m64
    • make
    • ./RLLibTest
  • 32-bit machines:
    • ./configure_m32
    • make
    • ./RLLibTest
  • Debugging:
    • ./configure_debug
    • make
    • ./RLLibTest

Visualization

RLLib provides a QT4.8 based Reinforcement Learning problems and algorithms visualization tool named RLLibViz. Currently RLLibViz visualizes following problems and algorithms:

  • On-policy:
    • SwingPendulum problem with continuous actions. We use AverageRewardActorCritic algorithm.
  • Off-policy:
    • ContinuousGridworld and MountainCar problems with discrete actions. We use Off-PAC algorithm.
  • In order to run the visualization tool, you need to have QT4.8 installed in your system.
  • In order to install RLLibViz:
    • Change directory to visualization/RLLibViz
    • ./configure
    • ./RLLibVizSwingPendulum
    • ./RLLibVizContinuousGridworld
    • ./RLLibVizMountainCar

Documentation

Publication

Dynamic Role Assignment using General ValueFunctions

Contact

Saminda Abeyruwan (saminda@cs.miami.edu)

Assets 2

RLLib: C++ Template Library to Learn Behaviors and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning

21 Oct 03:56
@samindaa samindaa
v1.0.2
6f77cda
Compare
Choose a tag to compare
RLLib: C++ Template Library to Learn Behaviors and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning

RLLib

(C++ Template Library to Learn Behaviors and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning)

RLLib is a lightweight C++ template library that implements incremental, standard, and gradient temporal-difference learning algorithms in Reinforcement Learning. It is a highly optimized library that is designed and written specifically for robotic applications. The implementation of the RLLib library is inspired by the RLPark API, which is a library of temporal-difference learning algorithms written in Java.

Features

  • Off-policy prediction algorithms:
    GTD(lambda)
    GQ(lambda)
  • Off-policy control algorithms:
    Greedy-GQ(lambda)
    Softmax-GQ(lambda)
    Off-PAC (can be used in on-policy setting)
  • On-policy algorithms:
    TD(lambda)
    SARSA(lambda)
    Expected-SARSA(lambda)
    Actor-Critic (natural, continuous and discrete actions, discounted, averaged reward settings, etc.)
  • Supervised learning algorithms:
    Adaline
    IDBD
    SemiLinearIDBD
    Autostep
  • Policies:
    Random
    Random50%Bias
    Greedy
    Epsilon-greedy
    Boltzmann
    Normal
    Softmax
  • Dot product:
    An efficient implementation of the dot product for tile coding based feature representations (with culling traces).
  • Benchmarking environments:
    Mountain Car
    Mountain Car 3D
    Swinging Pendulum
    Helicopter
    Continuous Grid World
  • Optimization:
    Optimized for very fast duty cycles (e.g., with culling traces, RLLib has been tested on the Robocup 3D simulator agent, and on the NAO V4 (cognition thread)).
  • Usage:
    The algorithm usage is very much similar to RLPark, therefore, swift learning curve.
  • Examples:
    There are a plethora of examples demonstrating on-policy and off-policy control experiments.
  • Visualization:
    We provide a Qt4 based application to visualize benchmark problems.

Demo

Off-PAC ContinuousGridworld
AverageRewardActorCritic SwingPendulum (Continuous Actions)

Usage

RLLib is a C++ template library. The header files are located in the src directly. You can simply include this directory from your projects, e.g., -I./src, to access the algorithms.

To access the control algorithms:

#include "ControlAlgorithm.h"

To access the predication algorithms:

#include "PredictorAlgorithm"

To access the supervised learning algorithms:

#include "SupervisedAlgorithm.h"

RLLib uses the namespace:

using namespace RLLib

Testing

RLLib provides a flexible testing framework. Follow these steps to quickly write a test case.

  • To access the testing framework: #include "HeaderTest.h"
#include "HeaderTest.h"

RLLIB_TEST(YourTest)

class YourTest Test: public YourTestBase
{
  public:
    YourTestTest() {}

    virtual ~Test() {}
    void run();

  private:
    void testYourMethod();
};

void YourTestBase::testYourMethod() {/** Your test code */}

void YourTestBase::run() { testYourMethod(); }
  • Add YourTest to the test/test.cfg file.

Test Configuration

The test cases are executed using:

./configure
make
./RLLibTest

Documentation

Publication

Dynamic Role Assignment using General ValueFunctions

Contact

Saminda Abeyruwan (saminda@cs.miami.edu)

Assets 2

RLLib: C++ Template Library to Learn Behaviors and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning

08 Oct 20:32
@samindaa samindaa
v1.0.1
d329994
Compare
Choose a tag to compare
RLLib: C++ Template Library to Learn Behaviors and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning

RLLib

(C++ Template Library to Learn Behaviors and Represent Learnable Knowledge using On/Off Policy Reinforcement Learning)

RLLib is a lightweight C++ template library that implements incremental, standard, and gradient temporal-difference learning algorithms in Reinforcement Learning. It is a highly optimized library that is designed and written specifically for robotic applications. The implementation of the RLLib library is inspired by the RLPark API, which is a library of temporal-difference learning algorithms written in Java.

Features

  • Off-policy prediction algorithms:
    GTD(lambda)
    GQ(lambda)
  • Off-policy control algorithms:
    Greedy-GQ(lambda)
    Softmax-GQ(lambda)
    Off-PAC (can be used in on-policy setting)
  • On-policy algorithms:
    TD(lambda)
    SARSA(lambda)
    Expected-SARSA(lambda)
    Actor-Critic (continuous and discrete actions, discounted, averaged reward settings, etc.)
  • Supervised learning algorithms:
    Adaline
    IDBD
    SemiLinearIDBD
    Autostep
  • Policies:
    Random
    Random50%Bias
    Greedy
    Epsilon-greedy
    Boltzmann
    Normal
    Softmax
  • Dot product:
    An efficient implementation of the dot product for tile coding based feature representations (with culling traces).
  • Benchmarking environments:
    Mountain Car
    Mountain Car 3D
    Swinging Pendulum
    Helicopter
    Continuous Grid World
  • Optimization:
    Optimized for very fast duty cycles (e.g., with culling traces, RLLib has been tested on the Robocup 3D simulator agent, and on the NAO V4 (cognition thread)).
  • Usage:
    The algorithm usage is very much similar to RLPark, therefore, swift learning curve.
  • Examples:
    There are a plethora of examples demonstrating on-policy and off-policy control experiments.
  • Visualization:
    We provide a Qt4 based application to visualize benchmark problems.

Usage

RLLib is a C++ template library. The header files are located in the src directly. You can simply include this directory from your projects, e.g., -I./src, to access the algorithms.

To access the control algorithms:

#include "ControlAlgorithm.h"

To access the predication algorithms:

#include "PredictorAlgorithm"

To access the supervised learning algorithms:

#include "SupervisedAlgorithm.h"

RLLib uses the namespace:

using namespace RLLib

Testing

RLLib provides a flexible testing framework. Follow these steps to quickly write a test case.

  • To access the testing framework: #include "HeaderTest.h"
#include "HeaderTest.h"

RLLIB_TEST(YourTest)

class YourTest Test: public YourTestBase
{
  public:
    YourTestTest() {}

    virtual ~Test() {}
    void run();

  private:
    void testYourMethod();
};

void YourTestBase::testYourMethod() {/** Your test code */}

void YourTestBase::run() { testYourMethod(); }
  • Add YourTest to the test/test.cfg file.

Test Configuration

The test cases are executed using:

./configure
make
./RLLibTest

Documentation

Contact

Saminda Abeyruwan (saminda@cs.miami.edu)

Assets 2