Python

Installing

Windows

Windows has wheels deployed to pypi for Python 3.6 (64-bit) and Python 3.7 (64-bit). Simply run pip install vowpalwabbit

OSX

1. Clone repo and Install dependencies
2. python setup.py install

Linux

The commands listed below will be using apt, replace this for the package manager used by your preferred flavor.

1. sudo apt update
2. sudo apt install libboost-dev libboost-program-options-dev libboost-system-dev libboost-math-dev libboost-test-dev libboost-python-dev zlib1g-dev cmake python3 python3-pip
3. pip3 install vowpalwabbit

Troubleshooting

Some common causes of failure for installation are due to missing or mismatched dependencies when Vowpal Wabbit builds. Make sure you have boost and boost-python installed on your system. Ensure you have all dependencies installed.

Windows

1. If pip install vowpalwabbit fails, ensure you are using the 64-bit version of Python 3.6 or 3.7 (The default download bitness for Windows is 32-bit)

2. If you need to build VowpalWabbit from source on Windows, use the following instructions

   1. Install vcpkg
   2. Install dependencies with vcpkg
      • Python 3.6 is built against commit aa095559
      • Python 3.7 is built against commit 8c3e093

   vcpkg --triplet x64-windows install zlib boost-system boost-program-options boost-test boost-align boost-foreach boost-python boost-math python3

   3. cd <repo_root>
   4. python setup.py --vcpkg-root C:\path\to\vcpkg install
      • Where C:\path\to\vcpkg is the root directory of where you cloned vcpkg

NOTE: Attempting to install boost-python in vcpkg while multiple python versions are installed in vcpkg will cause errors. Ensure only the relevant python version is installed in the environment.

OSX

There are several known issues regarding the VowpalWabbit installation for OSX.

1. Building and installing VowpalWabbit with Anaconda will crash on OSX 10.14 or later. See this issue for details
   • Uninstall Anaconda before attempting to install VowpalWabbit
2. CMake can't find Boost
   • Ensure the required Boost libraries are installed
   • If your Boost version is 1.70 or earlier, install using the command python setup.py --enable-boost-cmake install

Linux

For python3 on Ubuntu 16.04 LTS: Ubuntu 16.04 defaults to an old, custom-built version of boost. As such, the boost_python library names do not follow the standard naming convention adopted by offical boost releases for the boost_python libraries. You may need to manually create the relevant symlinks in this case. Example commands for python 3.5 follows:

$ cd /usr/lib/x86_64-linux-gnu/
$ sudo ln -s libboost_python-py35.so libboost_python3.so
$ sudo ln -s libboost_python-py35.a libboost_python3.a

Debugging Python/C++

The Python bindings can be debugged in a mixed mode fashion by connecting two debuggers. This lets you set breakpoints in both languages.

Requirements

• VSCode
• Python extension
• C/C++ extension

Steps

1. Edit launch.json to add the required targets
2. Ensure VW bindings are installed as debug 2. Right now this means editing line 88 of setup.py and setting config to 'debug' 3. Then run setup.py install -f under the repo root to install the bindings
3. Open the Python file to debug and run "Launch current Python file" debugger target
4. While the Python debugger is attached and broken at some breakpoint run "Attach GDB to Python" debugger target and select the Python process
   • If you get an error message like (I think this only happens in WSL):

     Error getting authority: Error initializing authority: Could not connect: No such file or directory
     [1] + Done(127)                  /usr/bin/pkexec "/usr/bin/gdb" --interpreter=mi --tty=${DbgTerm} 0<"/tmp/Microsoft-MIEngine-In-zxx2mqu5.eh4" 1>"/tmp/Microsoft-MIEngine-Out-3ynea04r.784"

   • Then run the following: echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope

Note: you cannot step over the language boundary. To move between you need to set breakpoints on either side and use "continue".

Documentation

Python documentation is automatically built and deployed on each commit to master using this GitHub Action.

Setup

The Python documentation is built using Sphinx and uses the numpydoc extension.

pip install sphinx numpydoc

Build

To run the Sphinx build:

cd python/docs/
make html

This will output the documentation into a build directory. The entry point will be at python/docs/build/html/index.html

Note: Documentation is built off the currently installed vowpalwabbit package. Therefore you must install your locally edited package to see edits in the documentation.

Resources

launch.json

{
  "name": "Launch current Python file",
  "type": "python",
  "request": "launch",
  "pythonPath": "/usr/bin/python",
  "program": "${file}",
  "cwd": "${workspaceFolder}",
  "console": "integratedTerminal",
  "justMyCode": false
},
{
  "name": "Attach GDB to Python",
  "type": "cppdbg",
  "request": "attach",
  "program": "/usr/bin/python",
  "processId": "${command:pickProcess}",
  "MIMode": "gdb",
  "setupCommands": [
       {
           "description": "Enable pretty-printing for gdb",
           "text": "-enable-pretty-printing",
           "ignoreFailures": true
       }
  ]
}

Links

• Original instructions
• Issue describing issue of using external terminal with WSL
• Issue describing elevation issue on WSL