Optimise LdaModel.show_topics performance to only call get_lambda once

[nicolaslecrique]

This function calls show_topic for each topic, which calls get_topic_terms which calls self.state.get_lambda().

self.state.get_lambda() call is very expensive, it is not dependent on a topic and can be called once for all.
On my setup (500 topics, 100k words, learned with 500k documents), calling show_topics takes 30sec, with 28sec inside the 500 get_lambda() calls

[souravsingh]

I would like to work on the issue. How do I start?

[tmylk]

Let's finish your other PRs first @souravsingh :)

[bhargavvader]

@tmylk I tried a fair amount to try and fix this but the only solutions which I could think of are not very pretty.

We could maybe have an option to pass the lambda values, so instead of calling it within the function each time, the matrix is just passed around and we can pick up the row we need.

The other option is to completely avoid calling show_topic in show_topics, and just write up code to do what is happening and keep self.state.get_lambda() only once in the beginning of the function. This also works, but is not really pretty, and there is code duplication.

Both the methods I tried work and pass tests without having to call get_lambda multiple times, but don't seem like the best solution, so I did not open a PR. If one of them sounds okay, I'll open a PR for it.

Any other suggestions?

[tmylk]

The optimised show_topics should not call the show_topic. That is a good solution

[nicolaslecrique]

Thanks a lot ! I'll check this