Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

MRG Feature stacker #1173

Merged
merged 11 commits into from

7 participants

Andreas Mueller Olivier Grisel Lars Gael Varoquaux Mathieu Blondel Alexandre Gramfort Vlad Niculae
Andreas Mueller
Owner

This estimator provides a Y piece for the pipeline.
I used it to combine word ngrams and char ngrams into a single transformer.
Basically it just concatenates the output of several transformers into one large feature.

If you think this is helpful, I'll add some docs and an example.
With this, together with Pipeline, one can build arbitrary complex graphs (with one source and one sink) of estimators in sklearn :)

TODO

  • tests
  • narrative documentation
  • example

Thanks to the awesome implementation of the BaseEstimator, grid search simply works - though with complicated graphs you get parameter names like feature_stacker__first_feature__feature_selection__percentile (more or less from my code ^^).

sklearn/linear_model/tests/test_randomized_l1.py
((13 lines not shown))
+
+ # center here because sparse matrices are usually not centered
+ X, y, _, _, _ = center_data(X, y, True, True)
+
+ X_sp = sparse.csr_matrix(X)
+
+ F, _ = f_classif(X, y)
+
+ scaling = 0.3
+ clf = RandomizedLogisticRegression(verbose=False, C=1., random_state=42,
+ scaling=scaling, n_resampling=50, tol=1e-3)
+ feature_scores = clf.fit(X, y).scores_
+ clf = RandomizedLogisticRegression(verbose=False, C=1., random_state=42,
+ scaling=scaling, n_resampling=50, tol=1e-3)
+ feature_scores_sp = clf.fit(X_sp, y).scores_
+ assert_equal(feature_scores, feature_scores_sp)
Olivier Grisel Owner
ogrisel added a note

This hunk seems to be unrelated to this PR.

Andreas Mueller Owner

whoops sorry, forked from wrong branch. just a sec.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Olivier Grisel
Owner

Very interesting. I want an example first! (then documentation and tests :)

Andreas Mueller
Owner

on it :)

Olivier Grisel
Owner

@amueller to avoid forking from non-master branches you should use something such as http://volnitsky.com/project/git-prompt/

sklearn/pipeline.py
((22 lines not shown))
+
+ def get_feature_names(self):
+ pass
+
+ def fit(self, X, y=None):
+ for name, trans in self.transformer_list:
+ trans.fit(X, y)
+ return self
+
+ def transform(self, X):
+ features = []
+ for name, trans in self.transformer_list:
+ features.append(trans.transform(X))
+ issparse = [sparse.issparse(f) for f in features]
+ if np.any(issparse):
+ features = sparse.hstack(features).tocsr()
Olivier Grisel Owner
ogrisel added a note

Maybe the tocsr() can be avoided. For instance the downstream model might prefer CSC such as ElasticNet for instance.

Lars Owner

Then again, bugs crop up every now and then where estimators that are supposed to handle any sparse format turn out to only handle CSR. It's a good defensive strategy to produce CSR by default (and it's unfortunate that sparse.hstack doesn't do this already).

Andreas Mueller Owner

I wrote this thing in the heat of the battle and I don't remember if there was a reason or if it was just a precaution. I'm inclined to think that I put it there because something, somewhere, broke.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Andreas Mueller
Owner

Yes, it should derive from transformer mixin.
@larsmans can I interpret your comments such that you think this is a good thing to have?

Andreas Mueller
Owner

Added a toy example.

Olivier Grisel
Owner

I think such a feature stack should provide some way to do feature group normalization in one way or another. But this probably require some experiments to know which normalization pattern is useful on such beast in practice.

Anybody has practical experience or insight to share on this?

Lars
Owner

GREAT idea! However, I don't like the name FeatureStacker much, as stacking implies putting things on top of each other, while this class concatenates things side-by-side.

I tried to find a "plumbing equivalent" of this class to keep with the pipeline metaphor, but I can't seem to find it. It's not quite a tee as it connects the various streams back together in the end. Maybe one of the other devs is more experienced with plumbing? :)

Olivier Grisel
Owner

BTW I think the example could be improved my using a less trivial example (e.g. using the digits dataset) and showing that the cross validate score best grid searched parameter set of the pipeline with stacked features is better than the pipeline with individual feature transformers separately.

Olivier Grisel
Owner

@larsmans maybe FeatureConcatenator?

Olivier Grisel
Owner

FeatureUnion?

Olivier Grisel ogrisel closed this
Lars larsmans reopened this
Lars
Owner

MultiTransformer?

Andreas Mueller
Owner
Gael Varoquaux
Andreas Mueller
Owner

I also like FeatureUnion.
Other possibilities: FeatureBinder, FeatureAssembler, FeatureCombiner.
Or maybe go away from feature? TransformerUnion, TransformBinder, TransformerBundle?

Hm i think I like TransformerBundle

Olivier Grisel
Owner

+1 for FeatureAssembler or FeatureUnion or TransformerBundle

Lars
Owner

+1 for TransformerBundle.

Andreas Mueller
Owner

In my application, I found the get_feature_names very helpful - I was using text data and some handcrafted features.
I fear in general this is hard to do. I thought about doing hasattr("get_feature_names") and otherwise just return estimator_name_0, estimator_name_1,.... This might be a bit problematic, though, as I don't think there is a reliable method to get the output dimensionality of a transformer :-/

Oh and @ogrisel for the normalization, each feature should be normalized separately, right?
This is "easily" possible but feeding the object pipelines of preprocessing and transformers. As normalization might be quite application specific, I think this solution is ok for the moment.
The code doesn't actually get too messy doing this.

Andreas Mueller
Owner

ugh I just tried to work on the example and noticed that #1034 wasn't in master yet.
Without a good way to look at the grid search results, this PR is a lot less useful I think.
Have to work on #1034 more :-/

Lars
Owner

We might introduce an n_features_out_ attribute/property on all transformers that work on feature vectors. For now, only supporting get_feature_names only when all underlying transformers do would be good enough, IMHO.

Andreas Mueller
Owner

@larsmans ok, will do that. Should be easy enough.

Andreas Mueller
Owner

Having a bit of a hard time creating a good example :-/

Olivier Grisel
Owner

Have you been able to use this kind of tool successfully for your kaggle contest? If so then we can stick to a simplistic toy example and tell in the narrative documentation which kind of feature bundle was proven useful in practice on which kind of problem (e.g. PCA feature + raw TF-IDF for text classification for instance).

Andreas Mueller
Owner

I can tell you how successful I was tomorrow ;)
It was definitely helpful to combine handcrafted features with word n-grams. Doing it using this estimator, I was still able to grid-seach for count-vectorize parameters such as min_df, ngram_range, etc. So that definitely helped.

sklearn/pipeline.py
@@ -199,3 +202,81 @@ def score(self, X, y=None):
def _pairwise(self):
# check if first estimator expects pairwise input
return getattr(self.steps[0][1], '_pairwise', False)
+
+
+class FeatureStacker(BaseEstimator, TransformerMixin):
+ """Concatenates results of multiple transformer objects.
+
+ This estimator applies a list of transformer objects in parallel to the
+ input data, then concatenates the results. This is useful to combine
+ several feature extraction mechanisms into a single estimator.
Mathieu Blondel Owner

single feature representation?

Andreas Mueller Owner

I prefer it the way it is, as getting the features out is not the important part, the important part is formulating it as an estimator.

Mathieu Blondel Owner

I misunderstood what you meant. Since you're talking about extraction mechanisms, it may be clearer to say "in a single transformer".

Andreas Mueller Owner

agreed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
sklearn/pipeline.py
((31 lines not shown))
+ for name, trans in self.transformer_list:
+ if not hasattr(trans, 'get_feature_names'):
+ raise AttributeError("Transformer %s does not provide"
+ " get_feature_names." % str(name))
+ feature_names.extend([name + "__" + f for f in trans.get_feature_names()])
+ return feature_names
+
+ def fit(self, X, y=None):
+ """Fit all transformers using X.
+
+ Parameters
+ ----------
+ X : array-like or sparse matrix, shape (n_samples, n_features)
+ Input data, used to fit transformers.
+ """
+ for name, trans in self.transformer_list:
Mathieu Blondel Owner

supporting n_jobs would be nice :)

Andreas Mueller Owner

In principle +1. Are there any transformers that use n_jobs? I am always afraid of having it on the wrong abstraction level....

Mathieu Blondel Owner

Since it is embarrassingly parallel and each transformer can take time to fit, I think supporting n_jobs would make sense.

Are there any transformers that use n_jobs?

Not that I know of but I hope that users have enough common sense to not enable n_jobs at two different levels :)

Andreas Mueller Owner

So I see you are of the optimist persuasion ;)
I'll add it.

Gael Varoquaux Owner
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
sklearn/pipeline.py
((4 lines not shown))
+
+
+class FeatureStacker(BaseEstimator, TransformerMixin):
+ """Concatenates results of multiple transformer objects.
+
+ This estimator applies a list of transformer objects in parallel to the
+ input data, then concatenates the results. This is useful to combine
+ several feature extraction mechanisms into a single estimator.
+
+ Parameters
+ ----------
+ transformers: list of (name, transformer)
+ List of transformer objects to be applied to the data.
+
+ """
+ def __init__(self, transformer_list):
Mathieu Blondel Owner

a transformer_weight option to give more importance to some transformers could be useful!

Andreas Mueller Owner

hm ok, why not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Mathieu Blondel
Owner

Nice idea indeed!

Andreas Mueller
Owner

@mblondel any votes on the name?

Mathieu Blondel
Owner

Some I like include FeatureAssembler, FeatureCombiner and FeatureUnion.

Andreas Mueller
Owner

Name votes:
FeatureAssembler II
FeatureCombiner I
FeatureUnion IIII
TransformerBundle III

(If I counted correctly, which is unlikely given my degree in math)
If no-one objects I'll rename to FeatureUnion and change the state of the PR to MRG.

Alexandre Gramfort
Owner
Andreas Mueller
Owner

Renamed, think this is good to go.

Andreas Mueller
Owner

Any more comments? (github claims this can not be merged but I just rebased, so it should be a fast-forward merge).

Olivier Grisel
Owner

This cannot be merged in master currently but appart from that +1 for merging :)

Gael Varoquaux

LGTM. :+1: for merge. Thanks @amueller !

Andreas Mueller amueller merged commit d087830 into from
Vlad Niculae
Owner

Thank you for this convenient transformer. In my application I had to hack it a bit, and I wonder whether the feature I wanted could be more generally useful.

Basically, sometimes you want to concatenate the same feature extractor multiple times, and have some of the parameters tied when grid searching.

In my case, I was learning a hyphenator, so my data points consist of 2 strings: the one to the left of the current position and the one to the right of the current position. For this I defined a ProjectionVectorizer that has a column attribute that just says "I only work on X[:, column]" and concatenated two of these. Now, when grid searching, it is common sense to use the same n-gram range for both transformers, so the cleanest way to do this was this quick hack (no error handling):

class HomogeneousFeatureUnion(FeatureUnion):
    def set_params(self, **params):
        for key, value in params.iteritems():
            for _, transf in self.transformer_list:
                transf.set_params(**{key: value})

This can be easily extended to support both tied params and specific params. I'm not sure whether I overengineered this, but I still have the feeling that this might pop up in other people's applications, so I wanted to raise the question.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
1  doc/modules/classes.rst
View
@@ -817,6 +817,7 @@ Pairwise metrics
:template: class.rst
pipeline.Pipeline
+ pipeline.FeatureUnion
.. _preprocessing_ref:
48 doc/modules/pipeline.rst
View
@@ -84,3 +84,51 @@ The pipeline has all the methods that the last estimator in the pipline has,
i.e. if the last estimator is a classifier, the :class:`Pipeline` can be used
as a classifier. If the last estimator is a transformer, again, so is the
pipeline.
+
+
+.. _feature_union:
+
+======================================
+FeatureUnion: Concatenating features
+======================================
+
+.. currentmodule:: sklearn.pipeline
+
+:class:`FeatureUnion` combines several transformer objects into a new
+transformer that combines their output. A :class:`FeatureUnion` takes
+a list of transformer objects. During fitting, each of these
+is fit to the data independently. For transforming data, the
+transformers are applied in parallel, and their output combined into a
+single output array or matrix.
+
+:class:`FeatureUnion` serves the same purposes as :class:`Pipeline` -
+convenience and joint parameter estimation and validation.
+
+:class:`FeatureUnion` and :class:`Pipeline` can be combined to
+create complex models.
+
+
+Usage
+=====
+
+The :class:`FeatureUnion` is build using a list of ``(key, value)`` pairs, where
+the ``key`` a string containing the name you want to give to a given transformation and ``value``
+is an estimator object::
+
+ >>> from sklearn.pipeline import FeatureUnion
+ >>> from sklearn.decomposition import PCA
+ >>> from sklearn.decomposition import KernelPCA
+ >>> estimators = [('linear_pca', PCA()), ('kernel_pca', KernelPCA())]
+ >>> combined = FeatureUnion(estimators)
+ >>> combined # doctest: +NORMALIZE_WHITESPACE
+ FeatureUnion(n_jobs=1, transformer_list=[('linear_pca', PCA(copy=True,
+ n_components=None, whiten=False)), ('kernel_pca', KernelPCA(alpha=1.0,
+ coef0=1, degree=3, eigen_solver='auto', fit_inverse_transform=False,
+ gamma=0, kernel='linear', max_iter=None, n_components=None, tol=0))],
+ transformer_weights=None)
+
+
+
+.. topic:: Examples:
+
+ * :ref:`example_feature_stacker.py`
10 doc/whats_new.rst
View
@@ -21,9 +21,13 @@ Changelog
- Speed up of :func:`metrics.precision_recall_curve` by Conrad Lee.
- Added support for reading/writing svmlight files with pairwise
- preference attribute (qid in svmlight file format) in
- :func:`datasets.dump_svmlight_file` and
- :func:`datasets.load_svmlight_file` by `Fabian Pedregosa`_.
+ preference attribute (qid in svmlight file format) in
+ :func:`datasets.dump_svmlight_file` and
+ :func:`datasets.load_svmlight_file` by `Fabian Pedregosa`_.
+
+ - New estimator :ref:`FeatureUnion <feature_union>` that concatenates results
+ of several transformers by `Andreas Müller`_.
+
API changes summary
-------------------
60 examples/feature_stacker.py
View
@@ -0,0 +1,60 @@
+"""
+=================================================
+Concatenating multiple feature extraction methods
+=================================================
+
+In many real-world examples, there are many ways to extract features from a
+dataset. Often it is benefitial to combine several methods to obtain good
+performance. This example shows how to use ``FeatureUnion`` to combine
+features obtained by PCA and univariate selection.
+
+Combining features using this transformer has the benefit that it allows
+cross validation and grid searches over the whole process.
+
+The combination used in this example is not particularly helpful on this
+dataset and is only used to illustrate the usage of FeatureUnion.
+"""
+
+# Author: Andreas Mueller <amueller@ais.uni-bonn.de>
+#
+# License: BSD 3-clause
+
+from sklearn.pipeline import Pipeline, FeatureUnion
+from sklearn.grid_search import GridSearchCV
+from sklearn.svm import SVC
+from sklearn.datasets import load_iris
+from sklearn.decomposition import PCA
+from sklearn.feature_selection import SelectKBest
+
+iris = load_iris()
+
+X, y = iris.data, iris.target
+
+# This dataset is way to high-dimensional. Better do PCA:
+pca = PCA(n_components=2)
+
+# Maybe some original features where good, too?
+selection = SelectKBest(k=1)
+
+# Build estimator from PCA and Univariate selection:
+
+combined_features = FeatureUnion([("pca", pca), ("univ_select", selection)])
+
+# Use combined features to transform dataset:
+X_features = combined_features.fit(X, y).transform(X)
+
+# Classify:
+svm = SVC(kernel="linear")
+svm.fit(X_features, y)
+
+# Do grid search over k, n_components and C:
+
+pipeline = Pipeline([("features", combined_features), ("svm", svm)])
+
+param_grid = dict(features__pca__n_components=[1, 2, 3],
+ features__univ_select__k=[1, 2],
+ svm__C=[0.1, 1, 10])
+
+grid_search = GridSearchCV(pipeline, param_grid=param_grid, verbose=10)
+grid_search.fit(X, y)
+print(grid_search.best_estimator_)
107 sklearn/pipeline.py
View
@@ -8,9 +8,13 @@
# Alexandre Gramfort
# Licence: BSD
-from .base import BaseEstimator
+import numpy as np
+from scipy import sparse
-__all__ = ['Pipeline']
+from .base import BaseEstimator, TransformerMixin
+from .externals.joblib import Parallel, delayed
+
+__all__ = ['Pipeline', 'FeatureUnion']
# One round of beers on me if someone finds out why the backslash
@@ -199,3 +203,102 @@ def score(self, X, y=None):
def _pairwise(self):
# check if first estimator expects pairwise input
return getattr(self.steps[0][1], '_pairwise', False)
+
+
+def _fit_one_transformer(transformer, X, y):
+ transformer.fit(X, y)
+
+
+def _transform_one(transformer, name, X, transformer_weights):
+ if transformer_weights is not None and name in transformer_weights:
+ # if we have a weight for this transformer, muliply output
+ return transformer.transform(X) * transformer_weights[name]
+ return transformer.transform(X)
+
+
+class FeatureUnion(BaseEstimator, TransformerMixin):
+ """Concatenates results of multiple transformer objects.
+
+ This estimator applies a list of transformer objects in parallel to the
+ input data, then concatenates the results. This is useful to combine
+ several feature extraction mechanisms into a single transformer.
+
+ Parameters
+ ----------
+ transformers: list of (name, transformer)
+ List of transformer objects to be applied to the data.
+
+ n_jobs: int, optional
+ Number of jobs to run in parallel (default 1).
+
+ transformer_weights: dict, optional
+ Multiplicative weights for features per transformer.
+ Keys are transformer names, values the weights.
+
+ """
+ def __init__(self, transformer_list, n_jobs=1, transformer_weights=None):
+ self.transformer_list = transformer_list
+ self.n_jobs = n_jobs
+ self.transformer_weights = transformer_weights
+
+ def get_feature_names(self):
+ """Get feature names from all transformers.
+
+ Returns
+ -------
+ feature_names : list of strings
+ Names of the features produced by transform.
+ """
+ feature_names = []
+ for name, trans in self.transformer_list:
+ if not hasattr(trans, 'get_feature_names'):
+ raise AttributeError("Transformer %s does not provide"
+ " get_feature_names." % str(name))
+ feature_names.extend([name + "__" + f
+ for f in trans.get_feature_names()])
+ return feature_names
+
+ def fit(self, X, y=None):
+ """Fit all transformers using X.
+
+ Parameters
+ ----------
+ X : array-like or sparse matrix, shape (n_samples, n_features)
+ Input data, used to fit transformers.
+ """
+ Parallel(n_jobs=self.n_jobs)(delayed(_fit_one_transformer)(trans, X, y)
+ for name, trans in self.transformer_list)
+ return self
+
+ def transform(self, X):
+ """Transform X separately by each transformer, concatenate results.
+
+ Parameters
+ ----------
+ X : array-like or sparse matrix, shape (n_samples, n_features)
+ Input data to be transformed.
+
+ Returns
+ -------
+ X_t : array-like or sparse matrix, shape (n_samples, sum_n_components)
+ hstack of results of transformers. sum_n_components is the
+ sum of n_components (output dimension) over transformers.
+ """
+ Xs = Parallel(n_jobs=self.n_jobs)(
+ delayed(_transform_one)(trans, name, X, self.transformer_weights)
+ for name, trans in self.transformer_list)
+ if any(sparse.issparse(f) for f in Xs):
+ Xs = sparse.hstack(Xs).tocsr()
+ else:
+ Xs = np.hstack(Xs)
+ return Xs
+
+ def get_params(self, deep=True):
+ if not deep:
+ return super(FeatureUnion, self).get_params(deep=False)
+ else:
+ out = dict(self.transformer_list)
+ for name, trans in self.transformer_list:
+ for key, value in trans.get_params(deep=True).iteritems():
+ out['%s__%s' % (name, key)] = value
+ return out
8 sklearn/tests/test_common.py
View
@@ -28,7 +28,7 @@
# import "special" estimators
from sklearn.grid_search import GridSearchCV
from sklearn.decomposition import SparseCoder
-from sklearn.pipeline import Pipeline
+from sklearn.pipeline import Pipeline, FeatureUnion
from sklearn.pls import _PLS, PLSCanonical, PLSRegression, CCA, PLSSVD
from sklearn.ensemble import BaseEnsemble
from sklearn.multiclass import OneVsOneClassifier, OneVsRestClassifier,\
@@ -45,9 +45,9 @@
SpectralClustering
from sklearn.linear_model import IsotonicRegression
-dont_test = [Pipeline, GridSearchCV, SparseCoder, EllipticEnvelope,
- EllipticEnvelop, DictVectorizer, LabelBinarizer, LabelEncoder,
- TfidfTransformer, IsotonicRegression]
+dont_test = [Pipeline, FeatureUnion, GridSearchCV, SparseCoder,
+ EllipticEnvelope, EllipticEnvelop, DictVectorizer, LabelBinarizer,
+ LabelEncoder, TfidfTransformer, IsotonicRegression]
meta_estimators = [BaseEnsemble, OneVsOneClassifier, OutputCodeClassifier,
OneVsRestClassifier, RFE, RFECV]
70 sklearn/tests/test_pipeline.py
View
@@ -2,17 +2,21 @@
Test the pipeline module.
"""
import numpy as np
+from scipy import sparse
from nose.tools import assert_raises, assert_equal, assert_false, assert_true
+from numpy.testing import assert_array_equal, \
+ assert_array_almost_equal
from sklearn.base import BaseEstimator, clone
-from sklearn.pipeline import Pipeline
+from sklearn.pipeline import Pipeline, FeatureUnion
from sklearn.svm import SVC
from sklearn.linear_model import LogisticRegression
from sklearn.feature_selection import SelectKBest, f_classif
from sklearn.decomposition.pca import PCA, RandomizedPCA
from sklearn.datasets import load_iris
from sklearn.preprocessing import StandardScaler
+from sklearn.feature_extraction.text import CountVectorizer
class IncorrectT(BaseEstimator):
@@ -174,3 +178,67 @@ def test_pipeline_methods_preprocessing_svm():
assert_equal(decision_function.shape, (n_samples, n_classes))
pipe.score(X, y)
+
+
+def test_feature_stacker():
+ # basic sanity check for feature stacker
+ iris = load_iris()
+ X = iris.data
+ X -= X.mean(axis=0)
+ y = iris.target
+ pca = RandomizedPCA(n_components=2)
+ select = SelectKBest(k=1)
+ fs = FeatureUnion([("pca", pca), ("select", select)])
+ fs.fit(X, y)
+ X_transformed = fs.transform(X)
+ assert_equal(X_transformed.shape, (X.shape[0], 3))
+
+ # check if it does the expected thing
+ assert_array_almost_equal(X_transformed[:, :-1], pca.fit_transform(X))
+ assert_array_equal(X_transformed[:, -1],
+ select.fit_transform(X, y).ravel())
+
+ # test if it also works for sparse input
+ X_sp = sparse.csr_matrix(X)
+ X_sp_transformed = fs.fit_transform(X_sp, y)
+ assert_array_almost_equal(X_transformed, X_sp_transformed.toarray())
+
+ # test setting parameters
+ fs.set_params(select__k=2)
+ assert_equal(fs.fit_transform(X, y).shape, (X.shape[0], 4))
+
+
+def test_feature_stacker_weights():
+ # test feature stacker with transformer weights
+ iris = load_iris()
+ X = iris.data
+ y = iris.target
+ pca = RandomizedPCA(n_components=2)
+ select = SelectKBest(k=1)
+ fs = FeatureUnion([("pca", pca), ("select", select)],
+ transformer_weights={"pca": 10})
+ fs.fit(X, y)
+ X_transformed = fs.transform(X)
+ # check against expected result
+ assert_array_almost_equal(X_transformed[:, :-1], 10 * pca.fit_transform(X))
+ assert_array_equal(X_transformed[:, -1],
+ select.fit_transform(X, y).ravel())
+
+
+def test_feature_stacker_feature_names():
+ JUNK_FOOD_DOCS = (
+ "the pizza pizza beer copyright",
+ "the pizza burger beer copyright",
+ "the the pizza beer beer copyright",
+ "the burger beer beer copyright",
+ "the coke burger coke copyright",
+ "the coke burger burger",
+ )
+ word_vect = CountVectorizer(analyzer="word")
+ char_vect = CountVectorizer(analyzer="char_wb", ngram_range=(3, 3))
+ ft = FeatureUnion([("chars", char_vect), ("words", word_vect)])
+ ft.fit(JUNK_FOOD_DOCS)
+ feature_names = ft.get_feature_names()
+ for feat in feature_names:
+ assert_true("chars__" in feat or "words__" in feat)
+ assert_equal(len(feature_names), 35)
Something went wrong with that request. Please try again.