Optimize imports

[federicotdn]

Proposed changes:
Fixes #3010.

The idea of this PR is to make the rasa module load faster inside Python. This is somewhat related to the time it takes to run rasa on the command line, but I consider that to be a separate, more complex, issue.

If we improve the load time of the rasa module, then people using our module directly will notice a slight speedup on their applications' start-up time, assuming they are not loading all sub-modules explicitly. If they are, the start-up time will not be affected.

To measure import time, first I considered just measuring the time it took to run python -c 'import rasa'. However, this wouldn't have been too precise since that would also measure the start-up time for the Python interpreter and built-in modules. A much better option was to use a new feature added in Python 3.7, importtime. With python -X importtime rasa, one can get a detailed report of how much time it took for the rasa module to be imported, and all of its sub-modules as well. I wrote a short script which runs the profiling command N times, selects the relevant parts of the output, and averages the results. It can be used with any number N and any Python module.

To find which modules it was most convenient to look at, I used tuna. Tuna will take the output of python -X importtime rasa and then display all imported modules and their import times on a flamegraph:

(import times for master branch)

To use tuna, I ran the following commands:

$ cd /to/the/rasa/repo
$ pip install tuna
$ python -X importtime -c 'import rasa' 2> imports.txt
$ tuna imports.txt

The changes I ended up applying to the base master branch were:

• In nlu/test.py, don't load CRFEntityExtractor early.
• In constants.py, remove the unused variable FALLBACK_CONFIG_PATH and then remove the pkg_resources import, which represented about 28% of the total loading time for the rasa module.
• In cli/utils.py, avoid loading the questionary library (unless type checking).
• In training_data/loading.py, avoid loading the requests library until it's used.

After applying these four changes, the total loading time for the rasa module went from an average of 0.545 seconds to 0.260 seconds (N=15). This represents approximately a 52.3% decrease in loading time. The absolute values in seconds may vary between hardware configurations and operating systems.

This PR also removes the rasa attribute from the rasa module, which was pointing at itself (Which allowed doing rasa.rasa.rasa.rasa.rasa...).

Status (please check what you already did):

• made PR ready for code review
• added some tests for the functionality
• updated the documentation
• updated the changelog
• reformat files using black (please check Readme for instructions)

[wochinge]

Great detectives work! Do you also have a screenshot of the new import times which we can add to the issue / pr? And can we create an issue to do the import optimization for the CLI as well?

[federicotdn]

Here's the new flamegraph for the updated imports: