LatentDirichletAllocation perplexity method broken in version 0.18.1 #7954

Closed
garyForeman opened this Issue Nov 30, 2016 · 4 comments

Comments

4 participants
@garyForeman
Contributor

garyForeman commented Nov 30, 2016

Description

The perplexity method of the LatentDirichletAllocation class appears to have broken during the transition from scikit-learn 0.17.1 to 0.18.1. The values returned by the method are no longer consistent with the values printed during training iterations (verbose=1, evaluate_every=1).

Steps/Code to Reproduce

Gist with reproducible example can be found here: https://gist.github.com/garyForeman/321a10ebe29215a0c1acbcb4b320fb8e

Expected Results

Final perplexity printed during training should equal the value returned by the perplexity method when passed the training data.

Results when using 0.17.1:
iteration: 100, perplexity: 4044.2226
Train set perplexity: 4044.22258392

Actual Results

Results when using 0.18.1:
iteration: 100, perplexity: 4042.6522
Train set perplexity: 7592353.46945

Versions

Darwin-15.6.0-x86_64-i386-64bit
('Python', '2.7.12 |Anaconda custom (x86_64)| (default, Jul 2 2016, 17:43:17) \n[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)]')
('NumPy', '1.11.2')
('SciPy', '0.18.1')
('Scikit-Learn', '0.18.1')

@amueller amueller added the Bug label Nov 30, 2016

@amueller amueller added this to the 0.19 milestone Nov 30, 2016

@garyForeman

This comment has been minimized.

Show comment
Hide comment
@garyForeman

garyForeman Dec 1, 2016

Contributor

Just want to add that I appreciate you looking into this, and thank you so much for your hard work in maintaining scikit-learn!

Contributor

garyForeman commented Dec 1, 2016

Just want to add that I appreciate you looking into this, and thank you so much for your hard work in maintaining scikit-learn!

@lesteve

This comment has been minimized.

Show comment
Hide comment
@lesteve

lesteve Dec 1, 2016

Member

Thanks for your detailed issue and the stand-alone example, just to say I can reproduce the problem.

Member

lesteve commented Dec 1, 2016

Thanks for your detailed issue and the stand-alone example, just to say I can reproduce the problem.

@garyForeman

This comment has been minimized.

Show comment
Hide comment
@garyForeman

garyForeman Dec 5, 2016

Contributor

I'll volunteer to contribute the fix, since I believe I've identified where the problem is. I'll likely want to reach out to discuss implementation specifics as needed. Thanks!

Contributor

garyForeman commented Dec 5, 2016

I'll volunteer to contribute the fix, since I believe I've identified where the problem is. I'll likely want to reach out to discuss implementation specifics as needed. Thanks!

@amueller

This comment has been minimized.

Show comment
Hide comment
@amueller

amueller Dec 5, 2016

Member

@garyForeman great, please open a pull-request to discuss there.

Member

amueller commented Dec 5, 2016

@garyForeman great, please open a pull-request to discuss there.

@jnothman jnothman closed this in #7992 Dec 20, 2016

jnothman added a commit that referenced this issue Dec 20, 2016

[MRG + 1] Fix perplexity method by adding _unnormalized_transform met…
…hod, Issue #7954 (#7992)

Also deprecate doc_topic_distr argument in perplexity method

sergeyf added a commit to sergeyf/scikit-learn that referenced this issue Feb 28, 2017

[MRG + 1] Fix perplexity method by adding _unnormalized_transform met…
…hod, Issue #7954 (#7992)

Also deprecate doc_topic_distr argument in perplexity method

Sundrique added a commit to Sundrique/scikit-learn that referenced this issue Jun 14, 2017

[MRG + 1] Fix perplexity method by adding _unnormalized_transform met…
…hod, Issue #7954 (#7992)

Also deprecate doc_topic_distr argument in perplexity method

NelleV added a commit to NelleV/scikit-learn that referenced this issue Aug 11, 2017

[MRG + 1] Fix perplexity method by adding _unnormalized_transform met…
…hod, Issue #7954 (#7992)

Also deprecate doc_topic_distr argument in perplexity method

paulha added a commit to paulha/scikit-learn that referenced this issue Aug 19, 2017

[MRG + 1] Fix perplexity method by adding _unnormalized_transform met…
…hod, Issue #7954 (#7992)

Also deprecate doc_topic_distr argument in perplexity method

maskani-moh added a commit to maskani-moh/scikit-learn that referenced this issue Nov 15, 2017

[MRG + 1] Fix perplexity method by adding _unnormalized_transform met…
…hod, Issue #7954 (#7992)

Also deprecate doc_topic_distr argument in perplexity method
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment