Skip to content

Commit

Permalink
added another two datasets
Browse files Browse the repository at this point in the history
  • Loading branch information
CDonnerer committed Jul 22, 2021
1 parent 8d94c0c commit 8c32069
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 11 deletions.
22 changes: 13 additions & 9 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -114,15 +114,19 @@ Benchmarking
Across a variety of datasets, ``XGBDistribution`` performs similarly to
``NGBRegressor``, but is substantially faster.

+--------------+------------------------------------+-----------------------------------+------------------------+
| | XGBDistribution | NGBRegressor | XGBRegressor |
+---------+----+-----------+-----------+------------+-----------+-----------+-----------+-----------+------------+
| Dataset | N | NLL | RMSE | Time (s) | NLL | RMSE | Time (s) | RMSE | Time (s) |
+=========+====+===========+===========+============+===========+===========+===========+===========+============+
| Boston |506 | 2.62±0.26 | 3.41±0.69 | 0.067±0.01 | 2.55±0.24 | 3.25±0.66 | 2.68±0.45 | 3.27±0.65 | 0.035±0.01 |
+---------+----+-----------+-----------+------------+-----------+-----------+-----------+-----------+------------+
| Concrete|1030| 3.14±0.21 | 5.41±0.74 | 0.13±0.03 | 3.09±0.13 | 5.62±0.69 | 5.79±0.59 | 4.38±0.70 | 0.09±0.02 |
+---------+----+-----------+-----------+------------+-----------+-----------+-----------+-----------+------------+
+---------------+------------------------------------+-----------------------------------+------------------------+
| | XGBDistribution | NGBRegressor | XGBRegressor |
+---------+-----+-----------+-----------+------------+-----------+-----------+-----------+-----------+------------+
| Dataset | N | NLL | RMSE | Time (s) | NLL | RMSE | Time (s) | RMSE | Time (s) |
+=========+=====+===========+===========+============+===========+===========+===========+===========+============+
| Boston |506 | 2.62±0.26 | 3.41±0.69 | 0.067±0.01 | 2.55±0.24 | 3.25±0.66 | 2.68±0.45 | 3.27±0.65 | 0.035±0.01 |
+---------+-----+-----------+-----------+------------+-----------+-----------+-----------+-----------+------------+
| Concrete|1030 | 3.14±0.21 | 5.41±0.74 | 0.13±0.03 | 3.09±0.13 | 5.62±0.69 | 5.79±0.59 | 4.38±0.70 | 0.09±0.02 |
+---------+-----+-----------+-----------+------------+-----------+-----------+-----------+-----------+------------+
| Energy |768 | 0.58±0.41 | 0.45±0.07 | 0.15±0.03 | 0.62±0.28 | 0.49±0.07 | 5.33±0.35 | 0.40±0.06 | 0.05±0.02 |
+---------+-----+-----------+-----------+------------+-----------+-----------+-----------+-----------+------------+
| Naval |11934|-5.11±0.06 | 0.0014(1) | 5.8±0.85 |-3.91±0.02 | 0.0059(1) | 43.6±0.5 | 0.00123(5)| 1.93±0.07 |
+---------+-----+-----------+-----------+------------+-----------+-----------+-----------+-----------+------------+

We used 10-fold cross-validation, where in each training fold 10% of the data
were split off as a validation set, repeated over 5 random seeds. All models were
Expand Down
4 changes: 2 additions & 2 deletions examples/benchmarking.py
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,7 @@ def ngb_regressor(data):

@evaluate
def xgb_distribution(data):
xgbd = XGBDistribution(max_depth=None, natural_gradient=True, n_estimators=500)
xgbd = XGBDistribution(max_depth=3, natural_gradient=True, n_estimators=500)
xgbd.fit(
data.X_train,
data.y_train,
Expand All @@ -239,7 +239,7 @@ def xgb_distribution(data):

@evaluate
def xgb_regressor(data):
xgb = XGBRegressor(max_depth=None, n_estimators=500)
xgb = XGBRegressor(max_depth=3, n_estimators=500)
xgb.fit(
data.X_train,
data.y_train,
Expand Down

0 comments on commit 8c32069

Please sign in to comment.