Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up importing #1070

Merged
merged 3 commits into from
Jul 1, 2018
Merged

Speed up importing #1070

merged 3 commits into from
Jul 1, 2018

Conversation

akx
Copy link
Contributor

@akx akx commented Jun 27, 2018

As discussed in #1069, importing tflearn can be very slow due to it importing all of tensorflow.contrib.

This PR significantly improves import speed:

(master) $ time python -c 'import tflearn'
        5.45 real         4.73 user         0.54 sys
(fast-import) $ time python -c 'import tflearn'
        1.37 real         1.17 user         0.19 sys

This is achieved by lazy-loading any tensorflow.contrib components that would otherwise be always loaded, with some caveats:

  • The first use of tflearn.initializations.xavier and .variance_scaling will be slow
  • tflearn.data_utils.VocabularyProcessor no longer inherits from tensorflow.contrib.learn.python.learn.preprocessing.text.VocabularyProcessor, which might break some isinstance() calls in user code.
  • The tensorflow/contrib/framework/python/ops/arg_scope.py file is vendored in tflearn.

Regarding VocabularyProcessor, though – it looks like the class did nothing more than wrap calls to its superclass, so maybe it should be removed altogether with a note to have people use the superclass directly instead (though it too is deprecated since TensorFlow 1.7.0).

Closes #1069

Copy link
Member

@aymericdamien aymericdamien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Thanks for helping with that. Importing contrib was indeed slowing down quite a lot.

@aymericdamien aymericdamien merged commit 29f08d1 into tflearn:master Jul 1, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants