/v1 does not work out of the box

[kedarmhaswade]

I cloned the repo and followed the instructions. The venv seems to have been installed alright. Following along the book, I wanted to play with the v1. When I click on the "Get Recommendations" button, I see this on the server console:

127.0.0.1 - - [01/Jan/2020 18:52:38] "GET /v1 HTTP/1.1" 200 -
Exception on /v1 [POST]
Traceback (most recent call last):
  File "/Users/me/gh/ml-powered-applications/ml_editor/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/me/gh/ml-powered-applications/ml_editor/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/me/gh/ml-powered-applications/ml_editor/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/me/gh/ml-powered-applications/ml_editor/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/Users/me/gh/ml-powered-applications/ml_editor/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/me/gh/ml-powered-applications/ml_editor/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/me/gh/ml-powered-applications/app.py", line 15, in v1
    return handle_text_request(request, "v1.html")
  File "/Users/me/gh/ml-powered-applications/app.py", line 31, in handle_text_request
    suggestions = get_recommendations_from_input(question)
  File "/Users/me/gh/ml-powered-applications/ml_editor/ml_editor.py", line 279, in get_recommendations_from_input
    tokenized_sentences = preprocess_input(processed)
  File "/Users/me/gh/ml-powered-applications/ml_editor/ml_editor.py", line 45, in preprocess_input
    sentences = nltk.sent_tokenize(text)
  File "/Users/me/gh/ml-powered-applications/ml_editor/lib/python3.7/site-packages/nltk/tokenize/__init__.py", line 105, in sent_tokenize
    tokenizer = load('tokenizers/punkt/{0}.pickle'.format(language))
  File "/Users/me/gh/ml-powered-applications/ml_editor/lib/python3.7/site-packages/nltk/data.py", line 868, in load
    opened_resource = _open(resource_url)
  File "/Users/me/gh/ml-powered-applications/ml_editor/lib/python3.7/site-packages/nltk/data.py", line 993, in _open
    return find(path_, path + ['']).open()
  File "/Users/me/gh/ml-powered-applications/ml_editor/lib/python3.7/site-packages/nltk/data.py", line 701, in find
    raise LookupError(resource_not_found)
LookupError:
**********************************************************************
  Resource punkt not found.
  Please use the NLTK Downloader to obtain the resource:

  >>> import nltk
  >>> nltk.download('punkt')

  For more information see: https://www.nltk.org/data.html

  Attempted to load tokenizers/punkt/PY3/english.pickle

  Searched in:
    - '/Users/me/nltk_data'
    - '/Users/me/gh/ml-powered-applications/ml_editor/bin/../nltk_data'
    - '/Users/me/gh/ml-powered-applications/ml_editor/bin/../share/nltk_data'
    - '/Users/me/gh/ml-powered-applications/ml_editor/bin/../lib/nltk_data'
    - '/usr/share/nltk_data'
    - '/usr/local/share/nltk_data'
    - '/usr/lib/nltk_data'
    - '/usr/local/lib/nltk_data'
    - ''
**********************************************************************

127.0.0.1 - - [01/Jan/2020 18:52:44] "POST /v1 HTTP/1.1" 500 -

Is there an instruction missing?
I made sure nltk is installed in the venv which is activated.

[hundredblocks]

Hi @kedarmhaswade

I've updated the repository with a few changes that should make this easier.

Unfortunately, in order for the nltk tokenizer to work properly, you need to install a few nltk packages. The only way to install these packages that I know of requires a process that is slightly more involved than pip install.

In order to fix the particular error you are encountering, you can take the following steps:

Start by activating the venv. Then, in the same console, launch a python session (just typing python should work)
Then, run the following two lines:
import nltk
nltk.download('punkt')

After you do this, the error should be fixed. You may get errors for other required nltk packages, and should be able to fix those in the same manner. I'll update the docs to clarify this process.