Skip to content
Branch: master
Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
datasets
LICENSE
README.md
imdb.py
imdb.sh
uwcse.py
uwcse.sh
v1-0.jar
webkb.py
webkb.sh

README.md

Experiments Subsection

This directory includes the scripts used to reproduce "Table 1."

All scripts are made available under the terms of the MIT License. A copy of the license is contained in this directory.

Setup

The results presented in this workshop were based on the srlearn==0.5.0 release.

pip install srlearn==0.5.0

Producing the results from Table 1

⚠️ Results were produced in macOS High Sierra with the following specs, small differences are likely to occur on.

  • Processor 3.4 GHz Intel Core i5
  • Memory 16 GB 2400 MHz DDR4
  • Graphics Radeon Pro 570 4096 MB

Column 1: srlearn (Python/Java)

Mean and standard deviation are calculated automatically.

$ python webkb.py
$ python imdb.py
$ python uwcse.py

Column 2: BoostSRL (Java/Shell)

Mean and standard deviation are not calculated automatically. They might be calculated from an interactive Python shell with numpy.

$ bash webkb.sh
$ bash imdb.sh
$ bash uwcse.sh

Datasets

Copies of the datasets are included in the datasets/ directory for completeness. All modes and parameters are held constant, the method for running (srlearn or BoostSRL) are varied.

Limitations

The general way that time is calculated for the BoostSRL numbers use Unix epoch time from the BSD DATE(1) command.

START_TIME=$(date +%s)
END_TIME=$(date +%s)
echo "$(($END_TIME - $START_TIME))"

Whereas the srlearn version is calculated with the time.perf_counter() function in the time library, leading to this general form:

_start = time.perf_counter()
_end = time.perf_counter()
print(_end - _start)

Each method has specific overheads that makes results imperfect for direct comparison.

You can’t perform that action at this time.