Skip to content

Commit

Permalink
rename outofbag_bootstrap (#266)
Browse files Browse the repository at this point in the history
  • Loading branch information
rasbt committed Oct 21, 2017
1 parent 4345289 commit 45f3507
Show file tree
Hide file tree
Showing 7 changed files with 24 additions and 24 deletions.
2 changes: 1 addition & 1 deletion docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,11 @@ pages:
- user_guide/data/wine_data.md
- evaluate:
- user_guide/evaluate/bootstrap.md
- user_guide/evaluate/BootstrapOutOufBag.md
- user_guide/evaluate/confusion_matrix.md
- user_guide/evaluate/lift_score.md
- user_guide/evaluate/mcnemar_table.md
- user_guide/evaluate/mcnemar.md
- user_guide/evaluate/OutOufBagBootstrap.md
- user_guide/evaluate/permutation_test.md
- user_guide/evaluate/scoring.md
- feature_extraction:
Expand Down
2 changes: 1 addition & 1 deletion docs/sources/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ The CHANGELOG for the current development version is available at
- Added a `loadings_` attribute to `PrincipalComponentAnalysis` to compute the factor loadings of the features on the principal components. [#251](https://github.com/rasbt/mlxtend/pull/251)
- Allow grid search over classifiers/regressors in ensemble and stacking estimators [#259](https://github.com/rasbt/mlxtend/pull/259)
- New `make_multiplexer_dataset` function that creates a dataset generated by a n-bit Boolean multiplexer for evaluating supervised learning algorithms [#263](https://github.com/rasbt/mlxtend/pull/263)
- Added a new `OutOfBagBootstrap` class, an implementation of the out-of-bag bootstrap to evaluate supervised learning algorithms [#265](https://github.com/rasbt/mlxtend/pull/265)
- Added a new `BootstrapOutOfBag` class, an implementation of the out-of-bag bootstrap to evaluate supervised learning algorithms [#265](https://github.com/rasbt/mlxtend/pull/265)

##### Changes

Expand Down
2 changes: 1 addition & 1 deletion docs/sources/USER_GUIDE_INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,11 @@

## `evaluate`
- [bootstrap](user_guide/evaluate/bootstrap.md)
- [BootstrapOutOfBag](user_guide/evaluate/BootstrapOutOfBag.md)
- [confusion_matrix](user_guide/evaluate/confusion_matrix.md)
- [lift_score](user_guide/evaluate/lift_score.md)
- [mcnemar_table](user_guide/evaluate/mcnemar_table.md)
- [mcnemar](user_guide/evaluate/mcnemar.md)
- [OutOufBagBootstrap](user_guide/evaluate/OutOfBagBootstrap.md)
- [permutation_test](user_guide/evaluate/permutation_test.md)
- [scoring](user_guide/evaluate/scoring.md)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# OutOfBagBootstrap"
"# BootstrapOutOfBag"
]
},
{
Expand All @@ -18,7 +18,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"> `from mlxtend.evaluate import OutOfBagBootstrap` "
"> `from mlxtend.evaluate import BootstrapOutOfBag` "
]
},
{
Expand All @@ -35,7 +35,7 @@
"Originally, the bootstrap method aims to determine the statistical properties of an estimator when the underlying distribution was unknown and additional samples are not available. Now, in order to exploit this method for the evaluation of predictive models, such as hypotheses for classification and regression, we may prefer a slightly different approach to bootstrapping using the so-called Out-Of-Bag (OOB) or Leave-One-Out Bootstrap (LOOB) technique. Here, we use out-of-bag samples as test sets for evaluation instead of evaluating the model on the training data. Out-of-bag samples are the unique sets of instances that are not used for model fitting as shown in the figure below [1].\n",
"\n",
"\n",
"![](OutOfBagBootstrap_files/bootrap_concept.png)\n",
"![](BootstrapOutOfBag_files/bootrap_concept.png)\n",
"\n",
"\n",
"The figure above illustrates how three random bootstrap samples drawn from an exemplary ten-sample dataset ($X_1,X_2, ..., X_{10}$) and their out-of-bag sample for testing may look like. In practice, Bradley Efron and Robert Tibshirani recommend drawing 50 to 200 bootstrap samples as being sufficient for reliable estimates [2]."
Expand All @@ -62,7 +62,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The `OutOfBagBootstrap` class mimics the behavior of scikit-learn's cross-validation classes, e.g., `KFold`:"
"The `BootstrapOutOfBag` class mimics the behavior of scikit-learn's cross-validation classes, e.g., `KFold`:"
]
},
{
Expand All @@ -81,11 +81,11 @@
}
],
"source": [
"from mlxtend.evaluate import OutOfBagBootstrap\n",
"from mlxtend.evaluate import BootstrapOutOfBag\n",
"import numpy as np\n",
"\n",
"\n",
"oob = OutOfBagBootstrap(n_splits=3)\n",
"oob = BootstrapOutOfBag(n_splits=3)\n",
"for train, test in oob.split(np.array([1, 2, 3, 4, 5])):\n",
" print(train, test)"
]
Expand All @@ -94,7 +94,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Consequently, we can use `OutOfBagBootstrap` objects via the `cross_val_score` method:"
"Consequently, we can use `BootstrapOutOfBag` objects via the `cross_val_score` method:"
]
},
{
Expand Down Expand Up @@ -137,7 +137,7 @@
}
],
"source": [
"print(cross_val_score(lr, X, y, cv=OutOfBagBootstrap(n_splits=3, random_seed=456)))"
"print(cross_val_score(lr, X, y, cv=BootstrapOutOfBag(n_splits=3, random_seed=456)))"
]
},
{
Expand All @@ -162,7 +162,7 @@
],
"source": [
"print('Mean accuracy: %.1f%%' % np.mean(100*cross_val_score(\n",
" lr, X, y, cv=OutOfBagBootstrap(n_splits=200, random_seed=456))))"
" lr, X, y, cv=BootstrapOutOfBag(n_splits=200, random_seed=456))))"
]
},
{
Expand Down Expand Up @@ -206,7 +206,7 @@
}
],
"source": [
"accuracies = cross_val_score(lr, X, y, cv=OutOfBagBootstrap(n_splits=1000, random_seed=456))\n",
"accuracies = cross_val_score(lr, X, y, cv=BootstrapOutOfBag(n_splits=1000, random_seed=456))\n",
"mean = np.mean(accuracies)\n",
"\n",
"lower = np.percentile(accuracies, 2.5)\n",
Expand Down Expand Up @@ -243,9 +243,9 @@
"name": "stdout",
"output_type": "stream",
"text": [
"## OutOfBagBootstrap\n",
"## BootstrapOutOfBag\n",
"\n",
"*OutOfBagBootstrap(n_splits=200, random_seed=None)*\n",
"*BootstrapOutOfBag(n_splits=200, random_seed=None)*\n",
"\n",
"**Parameters**\n",
"\n",
Expand Down Expand Up @@ -328,7 +328,7 @@
}
],
"source": [
"with open('../../api_modules/mlxtend.evaluate/OutOfBagBootstrap.md', 'r') as f:\n",
"with open('../../api_modules/mlxtend.evaluate/BootstrapOutOfBag.md', 'r') as f:\n",
" s = f.read() \n",
"print(s)"
]
Expand Down
4 changes: 2 additions & 2 deletions mlxtend/evaluate/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@
from .mcnemar import mcnemar_table
from .mcnemar import mcnemar
from .bootstrap import bootstrap
from .outofbag_bootstrap import OutOfBagBootstrap
from .bootstrap_outofbag import BootstrapOutOfBag
from .permutation import permutation_test


__all__ = ["scoring", "confusion_matrix",
"mcnemar_table", "mcnemar", "lift_score",
"bootstrap", "permutation_test", "OutOfBagBootstrap"]
"bootstrap", "permutation_test", "BootstrapOutOfBag"]
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
import numpy as np


class OutOfBagBootstrap(object):
class BootstrapOutOfBag(object):
"""
Parameters
----------
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,18 @@
# License: BSD 3 clause

import numpy as np
from mlxtend.evaluate import OutOfBagBootstrap
from mlxtend.evaluate import BootstrapOutOfBag
from mlxtend.utils import assert_raises


def test_defaults():
oob = OutOfBagBootstrap()
oob = BootstrapOutOfBag()
results = list(oob.split(np.array([1, 2, 3, 4, 5])))
assert len(results) == 200


def test_splits():
oob = OutOfBagBootstrap(n_splits=3, random_seed=123)
oob = BootstrapOutOfBag(n_splits=3, random_seed=123)
results = list(oob.split(np.array([1, 2, 3, 4, 5])))
assert len(results) == 3
assert np.array_equal(results[0][0], np.array([2, 4, 2, 1, 3]))
Expand All @@ -28,10 +28,10 @@ def test_splits():
def test_invalid_splits():
assert_raises(ValueError,
'Number of splits must be greater than 1.',
OutOfBagBootstrap,
BootstrapOutOfBag,
0)


def test_get_n_splits():
oob = OutOfBagBootstrap(n_splits=3, random_seed=123)
oob = BootstrapOutOfBag(n_splits=3, random_seed=123)
assert oob.n_splits == 3

0 comments on commit 45f3507

Please sign in to comment.