TST Need tests for multilabel format issues #2022

Closed
jnothman opened this Issue Jun 2, 2013 · 7 comments

Projects

None yet

4 participants

@jnothman
Member
jnothman commented Jun 2, 2013

In sklearn/tests/test_multiclass.py we should be testing for:

  • identical results given: sequence of sequences, binarized with neg_label=0, binarized with neg_label=-1
  • the edge case where all entries have the same number of labels
  • ensure it works with sub-samples, via cross_val_score

(Writing the tests should be easy; making them pass may not be.)

@arjoly
Member
arjoly commented Jun 3, 2013

ensure it works with sub-samples, via cross_val_score

I don't think it will work for sub-samples of mix sequences of sequences and binary indicator format.
If one label is missing from the sequence of sequences input, you won't be able to infer correctly
the unique set of labels. There won't be a unique mapping see #2015.

@jnothman
Member
jnothman commented Jun 3, 2013

I don't understand you, because cross_val_score can't take mixed formats. It takes a single y and essentially performs [est.fit(X[train], y[train]).score(X[test], y[test]) for train, test in cv]. I had thought for a moment you were right that there were problems with sub-samples missing labels, but as long as every estimator accepting sequence of sequence input also outputs in that format (just as multiclass classifiers transform their output with classes_), it's fine.

@arjoly
Member
arjoly commented Jun 3, 2013

You are right.

@amueller
Member

no code was written yet, retagging.

@GaelVaroquaux
Member

No code, retagging, there is no point trying to rush that.

@arjoly
Member
arjoly commented Jul 19, 2014

Since we are dropping support for sequence of sequence, do you still want those tests for sequence of sequences?

@jnothman
Member

Nahh.

On 20 July 2014 00:27, Arnaud Joly notifications@github.com wrote:

Since we are dropping support for sequence of sequence, do you still
want those tests for sequence of sequences?


Reply to this email directly or view it on GitHub
#2022 (comment)
.

@jnothman jnothman closed this Jul 19, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment