Skip to content
Predict/log/learn/update reinforcement learning loop
C++ C# C CMake Python Shell Other
Branch: master
Clone or download

Latest commit

lokitoth Update VW Version to a496fb9 (#128)
_Moves from [cb914a038fdb238bc577257b3328c63e02ca74aa](VowpalWabbit/vowpal_wabbit@cb914a0) to [a496fb9a0de250ca1dab9cb8d7bc3827e1ce8de1](VowpalWabbit/vowpal_wabbit@a496fb9

Updates:
* Add chain hash option (VowpalWabbit/vowpal_wabbit#2214)
* Allow overriding --epsilon value from model file (VowpalWabbit/vowpal_wabbit#2277)
* Deprecate "-q:" (VowpalWabbit/vowpal_wabbit#2252)
* Distributionally Robust CB (VowpalWabbit/vowpal_wabbit#2203)

Fixes:
* Support interpreting the string NaN as a float nan in dsjson (VowpalWabbit/vowpal_wabbit#2256)
   > This prevents a crash in this case
* Multiline parsing in python (VowpalWabbit/vowpal_wabbit#2259)
* Fix segfault when reading from cache, using interactions and audit (VowpalWabbit/vowpal_wabbit#2250)
   > -q :: is the correct way to do this, now
* Performance fixes (VowpalWabbit/vowpal_wabbit#2220)
* Reduce probability of overflow (VowpalWabbit/vowpal_wabbit#2217)
* Upgrade Newtonsoft.Json in vw_cli (VowpalWabbit/vowpal_wabbit#2215)
* Fix delete behavior for examples created using parse function (VowpalWabbit/vowpal_wabbit#2206)
* Fix predict affecting model state when in library mode (VowpalWabbit/vowpal_wabbit#2190)

+ Other smaller fixes
Latest commit df162dc Feb 20, 2020

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.nuget Add cmake build definitions for reinforcement_learning (#3) Nov 5, 2018
.pipelines Create Linux and MacOS pipeline on Azure DevOps (#96) Sep 4, 2019
.scripts
bindings Fix SafeHandle memory safety (#116) Jan 9, 2020
cmake/Modules Rajan/preamble fb (#24) Dec 6, 2018
doc Fix python bindings and update version to 0.1.2 (#77) May 24, 2019
examples Ataymano/vw88 upgrade (#122) Dec 30, 2019
ext_libs
include Conditional contextual bandit (#84) Dec 20, 2019
rlclientlib Update VW Version to a496fb9 (#128) Feb 20, 2020
test_tools
unit_test
.gitignore strong name signing changes (#74) May 27, 2019
.gitmodules Update submodule repo (#6) Nov 6, 2018
.travis.yml Add travis build script (#2) Oct 12, 2018
CMakeLists.txt
LICENSE.txt Add vowpal_wabbit submodule and fix build scripts to work with new re… Oct 5, 2018
README.md
appveyor.yml strong name signing changes (#74) May 27, 2019
build-linux.sh Cleanup cmake files for devirtualization by using a pragma in header … Nov 6, 2019
build-windows.cmd Split build pipeline into per-stage scripts (#50) Mar 5, 2019
rl.sln Conditional contextual bandit (#84) Dec 20, 2019

README.md

Linux Build Status) MacOS Build Status Windows Build status

reinforcement_learning

git submodule update --init --recursive

Ubuntu

Dependencies

sudo apt-get install libboost-all-dev libssl-dev

Install Cpprest

cd ~
git clone https://github.com/Microsoft/cpprestsdk.git cpprestsdk
cd cpprestsdk
# Checkout 2.10.1 version of cpprestsdk
git checkout e8dda215426172cd348e4d6d455141f40768bf47
git submodule update --init
cd Release
mkdir build
cd build
cmake ..
make -j `nproc`
make install
cd ../../../
rm -rf cpprestsdk

Install Flatbuffers

cd ~
git clone https://github.com/google/flatbuffers.git flatbuffers
cd ./flatbuffers
# 1.10.0 release commit
git checkout 925c1d77fcc72636924c3c13428a34180c30f96f
mkdir build
cd build
cmake .. -DFLATBUFFERS_BUILD_TESTS=Off -DFLATBUFFERS_INSTALL=On -DCMAKE_BUILD_TYPE=Release -DFLATBUFFERS_BUILD_FLATHASH=Off
make -j `nproc`
sudo make install
cd ../../
rm -rf flatbuffers

Build

mkdir build
cd build
cmake ..
make

Troubleshooting

If you get an error similar to the following on MacOS when running cmake .., then you may be able to fix it by supplying the OpenSSL path to CMake.

Make Error at /usr/local/Cellar/cmake/3.14.4/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the
  system variable OPENSSL_ROOT_DIR (missing: OPENSSL_INCLUDE_DIR)
Call Stack (most recent call first):
  /usr/local/Cellar/cmake/3.14.4/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/Cellar/cmake/3.14.4/share/cmake/Modules/FindOpenSSL.cmake:413 (find_package_handle_standard_args)
  /usr/local/Cellar/cmake/3.14.4/share/cmake/Modules/CMakeFindDependencyMacro.cmake:48 (find_package)
  /usr/local/lib/cpprestsdk/cpprestsdk-config.cmake:11 (find_dependency)
  CMakeLists.txt:9 (find_package)

This can be fixed by invoking CMake similar to the following:

cmake -DOPENSSL_ROOT_DIR=/usr/local/Cellar/openssl/1.0.2r -DOPENSSL_LIBRARIES=/usr/local/Cellar/openssl/1.0.2r/lib ..

Installing cpprestsdk on Ubuntu18.04 using apt-get may result in cmake failing with:

CMake Error at CMakeLists.txt:9 (find_package):
  By not providing "Findcpprestsdk.cmake" in CMAKE_MODULE_PATH this project
  has asked CMake to find a package configuration file provided by
  "cpprestsdk", but CMake did not find one.

  Could not find a package configuration file provided by "cpprestsdk" with
  any of the following names:

    cpprestsdkConfig.cmake
    cpprestsdk-config.cmake

  Add the installation prefix of "cpprestsdk" to CMAKE_PREFIX_PATH or set
  "cpprestsdk_DIR" to a directory containing one of the above files.  If
  "cpprestsdk" provides a separate development package or SDK, be sure it has
  been installed.

The workaround is to specify where to search

cmake .. -DCMAKE_PREFIX_PATH=/usr/lib/x86_64-linux-gnu/cmake

Test

mkdir build
cd build
cmake ..
make rltest
make test

Windows

Windows dependencies are managed through Vcpkg and Nuget.

vcpkg install cpprestsdk:x64-windows
vcpkg install flatbuffers:x64-windows

You'll need to add the flatbuffers tool directory to your PATH aswell: <vcpkg_root>\installed\x64-windows\tools\flatbuffers

Build + Test

Set VcpkgIntegration environment variable to vcpkg.targets file on your machine

Example: VcpkgIntegration=c:\s\vcpkg\scripts\buildsystems\msbuild\vcpkg.targets

Ensure that the v140 toolset is installed in Visual Studio 2017.

Open rl.sln in Visual Studio 2017. Build Release or Debug x64 configuration.

Experimental - CMake on Windows (Do not mix with checked in solution)

Using CMake is an alternate way to configure and build the project. Currently it only supports the C++ projects. All dependencies are managed through Vcpkg:

# Warning only use if generating solution with CMake)
vcpkg install cpprestsdk:x64-windows
vcpkg install zlib:x64-windows
vcpkg install boost-system:x64-windows
vcpkg install boost-program-options:x64-windows
vcpkg install boost-test:x64-windows
vcpkg install boost-uuid:x64-windows

Run the following to generate a solution file to open in Visual Studio 2017:

mkdir build
cd build
cmake .. -G "Visual Studio 15 2017 Win64" -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x64-windows
.\reinforcement_learning.sln

Make targets

  • doc - Python and C++ docs
  • _rl_client - Python bindings
  • rlclientlib - rlclient library
  • rltest - unit tests
You can’t perform that action at this time.