Tutorial_1_Sentence_classification_with_word_embeddings.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Tutorial_1_Sentence_classification_with_word_embeddings.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": []
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.5"
    },
    "kernelspec": {
      "display_name": "Py3.6",
      "language": "python",
      "name": "py36"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "cZRyumBocHJF"
      },
      "source": [
        "# Tutorial 1.  Sentence classification with word embeddings\n",
        "\n",
        "This tutorial is aimed to make participants familiar with text classification on **DeepPavlov**.\n",
        "\n",
        "The tutorial has the following **structure**:\n",
        "\n",
        "1. [Library and requirements installation](#Library-and-requirements-installation)\n",
        "\n",
        "2. [Dataset downloading](#Dataset-downloading)\n",
        "\n",
        "3. [Dataset Reader](#Dataset-Reader): [docs link](https://deeppavlov.readthedocs.io/en/latest/apiref/dataset_readers.html)\n",
        "\n",
        "4. [Dataset Iterator](#Dataset-Iterator): [docs link](https://deeppavlov.readthedocs.io/en/latest/apiref/dataset_iterators.html)\n",
        "\n",
        "5. [Preprocessor](#Preprocessor): [docs link](https://deeppavlov.readthedocs.io/en/latest/components/data_processors.html)\n",
        "\n",
        "6. [Tokenizer](#Tokenizer): [docs link](https://deeppavlov.readthedocs.io/en/latest/components/data_processors.html)\n",
        "\n",
        "7. [GloVe Embedder](#Embedder): [docs link](https://deeppavlov.readthedocs.io/en/latest/components/data_processors.html)\n",
        "[pre-trained embeddings link](https://deeppavlov.readthedocs.io/en/latest/intro/pretrained_vectors.html)\n",
        "\n",
        "8. [Vocabulary of classes](#Vocabulary-of-classes)\n",
        "\n",
        "9. [Keras Classifier](#Classifier): [docs link](https://deeppavlov.readthedocs.io/en/latest/components/classifiers.html)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "0wFSAIVwcHJt"
      },
      "source": [
        "## Library and requirements installation\n",
        "\n",
        "Let's install `DeepPavlov` library and dependencies for Keras classification model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "82JKwt5_cHJu",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "68e1a748-76c0-4b30-af8c-041410868289"
      },
      "source": [
        "!pip install deeppavlov"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting deeppavlov\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/03/4f/1f73825653f388ead9a9ea2f46cad51c92bd84a899ebf983906013e14d1c/deeppavlov-0.4.0-py3-none-any.whl (682kB)\n",
            "\u001b[K     |████████████████████████████████| 686kB 2.7MB/s \n",
            "\u001b[?25hCollecting Cython==0.28.5 (from deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/19/8e/32b280abb0947a96cdbb8329fb2014851a21fc1d099009f946ea8a8202c3/Cython-0.28.5-cp36-cp36m-manylinux1_x86_64.whl (3.4MB)\n",
            "\u001b[K     |████████████████████████████████| 3.4MB 39.9MB/s \n",
            "\u001b[?25hCollecting numpy==1.14.5 (from deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/68/1e/116ad560de97694e2d0c1843a7a0075cc9f49e922454d32f49a80eb6f1f2/numpy-1.14.5-cp36-cp36m-manylinux1_x86_64.whl (12.2MB)\n",
            "\u001b[K     |████████████████████████████████| 12.2MB 34.4MB/s \n",
            "\u001b[?25hRequirement already satisfied: h5py==2.8.0 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (2.8.0)\n",
            "Collecting pytelegrambotapi==3.5.2 (from deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/a4/5a/7aab147b253f19e5ef007316f39cf693a63d5cd7f654c3805c76f6bde979/pyTelegramBotAPI-3.5.2.tar.gz (51kB)\n",
            "\u001b[K     |████████████████████████████████| 61kB 19.1MB/s \n",
            "\u001b[?25hCollecting keras==2.2.0 (from deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/68/12/4cabc5c01451eb3b413d19ea151f36e33026fc0efb932bf51bcaf54acbf5/Keras-2.2.0-py2.py3-none-any.whl (300kB)\n",
            "\u001b[K     |████████████████████████████████| 307kB 40.1MB/s \n",
            "\u001b[?25hCollecting flask==1.0.2 (from deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/7f/e7/08578774ed4536d3242b14dacb4696386634607af824ea997202cd0edb4b/Flask-1.0.2-py2.py3-none-any.whl (91kB)\n",
            "\u001b[K     |████████████████████████████████| 92kB 24.0MB/s \n",
            "\u001b[?25hCollecting pymorphy2-dicts-ru (from deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/7c/9b/358faaff410f65a4ad159275e897b5956dcb20576c5b8e764b971c1634d7/pymorphy2_dicts_ru-2.4.404381.4453942-py2.py3-none-any.whl (8.0MB)\n",
            "\u001b[K     |████████████████████████████████| 8.0MB 25.7MB/s \n",
            "\u001b[?25hCollecting scikit-learn==0.19.1 (from deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/3d/2d/9fbc7baa5f44bc9e88ffb7ed32721b879bfa416573e85031e16f52569bc9/scikit_learn-0.19.1-cp36-cp36m-manylinux1_x86_64.whl (12.4MB)\n",
            "\u001b[K     |████████████████████████████████| 12.4MB 27.6MB/s \n",
            "\u001b[?25hCollecting pandas==0.23.1 (from deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/57/eb/6ab533ea8e35e7dd159af6922ac1123d4565d89f3926ad9a6aa46530978f/pandas-0.23.1-cp36-cp36m-manylinux1_x86_64.whl (11.8MB)\n",
            "\u001b[K     |████████████████████████████████| 11.8MB 37.5MB/s \n",
            "\u001b[?25hCollecting scipy==1.1.0 (from deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/a8/0b/f163da98d3a01b3e0ef1cab8dd2123c34aee2bafbb1c5bffa354cc8a1730/scipy-1.1.0-cp36-cp36m-manylinux1_x86_64.whl (31.2MB)\n",
            "\u001b[K     |████████████████████████████████| 31.2MB 1.6MB/s \n",
            "\u001b[?25hCollecting requests==2.19.1 (from deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/65/47/7e02164a2a3db50ed6d8a6ab1d6d60b69c4c3fdf57a284257925dfc12bda/requests-2.19.1-py2.py3-none-any.whl (91kB)\n",
            "\u001b[K     |████████████████████████████████| 92kB 23.1MB/s \n",
            "\u001b[?25hRequirement already satisfied: nltk==3.2.5 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (3.2.5)\n",
            "Collecting fuzzywuzzy==0.16.0 (from deeppavlov)\n",
            "  Downloading https://files.pythonhosted.org/packages/3b/36/be990a35c7e8ed9dc176c43b5699cd971cec0b6f9ef858843374171df4f2/fuzzywuzzy-0.16.0-py2.py3-none-any.whl\n",
            "Collecting flask-cors==3.0.6 (from deeppavlov)\n",
            "  Downloading https://files.pythonhosted.org/packages/d1/db/f3495569d5c3e2bdb9fb8a66c54503364abb6f35a9da2227cf5c9c50dc42/Flask_Cors-3.0.6-py2.py3-none-any.whl\n",
            "Collecting tqdm==4.23.4 (from deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/93/24/6ab1df969db228aed36a648a8959d1027099ce45fad67532b9673d533318/tqdm-4.23.4-py2.py3-none-any.whl (42kB)\n",
            "\u001b[K     |████████████████████████████████| 51kB 15.1MB/s \n",
            "\u001b[?25hCollecting flasgger==0.9.1 (from deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/f7/4c/8735be27bebb88b0acbdc9db1d522583db10821aec3d3fb6112df0f41701/flasgger-0.9.1-py2.py3-none-any.whl (4.1MB)\n",
            "\u001b[K     |████████████████████████████████| 4.1MB 34.3MB/s \n",
            "\u001b[?25hCollecting overrides==1.9 (from deeppavlov)\n",
            "  Downloading https://files.pythonhosted.org/packages/de/55/3100c6d14c1ed177492fcf8f07c4a7d2d6c996c0a7fc6a9a0a41308e7eec/overrides-1.9.tar.gz\n",
            "Collecting pymorphy2==0.8 (from deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/a3/33/fff9675c68b5f6c63ec8c6e6ff57827dda28a1fa5b2c2d727dffff92dd47/pymorphy2-0.8-py2.py3-none-any.whl (46kB)\n",
            "\u001b[K     |████████████████████████████████| 51kB 18.2MB/s \n",
            "\u001b[?25hCollecting rusenttokenize==0.0.4 (from deeppavlov)\n",
            "  Downloading https://files.pythonhosted.org/packages/f8/4c/e2aeee9cdcc266303289ad8c4acfdcf401781646bcc311ee2bf18f84d663/rusenttokenize-0.0.4-py3-none-any.whl\n",
            "Collecting pyopenssl==18.0.0 (from deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/96/af/9d29e6bd40823061aea2e0574ccb2fcf72bfd6130ce53d32773ec375458c/pyOpenSSL-18.0.0-py2.py3-none-any.whl (53kB)\n",
            "\u001b[K     |████████████████████████████████| 61kB 21.2MB/s \n",
            "\u001b[?25hRequirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from h5py==2.8.0->deeppavlov) (1.12.0)\n",
            "Collecting keras-preprocessing==1.0.1 (from keras==2.2.0->deeppavlov)\n",
            "  Downloading https://files.pythonhosted.org/packages/f8/33/275506afe1d96b221f66f95adba94d1b73f6b6087cfb6132a5655b6fe338/Keras_Preprocessing-1.0.1-py2.py3-none-any.whl\n",
            "Collecting keras-applications==1.0.2 (from keras==2.2.0->deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/e2/60/c557075e586e968d7a9c314aa38c236b37cb3ee6b37e8d57152b1a5e0b47/Keras_Applications-1.0.2-py2.py3-none-any.whl (43kB)\n",
            "\u001b[K     |████████████████████████████████| 51kB 18.2MB/s \n",
            "\u001b[?25hRequirement already satisfied: pyyaml in /usr/local/lib/python3.6/dist-packages (from keras==2.2.0->deeppavlov) (3.13)\n",
            "Requirement already satisfied: itsdangerous>=0.24 in /usr/local/lib/python3.6/dist-packages (from flask==1.0.2->deeppavlov) (1.1.0)\n",
            "Requirement already satisfied: click>=5.1 in /usr/local/lib/python3.6/dist-packages (from flask==1.0.2->deeppavlov) (7.0)\n",
            "Requirement already satisfied: Werkzeug>=0.14 in /usr/local/lib/python3.6/dist-packages (from flask==1.0.2->deeppavlov) (0.15.4)\n",
            "Requirement already satisfied: Jinja2>=2.10 in /usr/local/lib/python3.6/dist-packages (from flask==1.0.2->deeppavlov) (2.10.1)\n",
            "Requirement already satisfied: pytz>=2011k in /usr/local/lib/python3.6/dist-packages (from pandas==0.23.1->deeppavlov) (2018.9)\n",
            "Requirement already satisfied: python-dateutil>=2.5.0 in /usr/local/lib/python3.6/dist-packages (from pandas==0.23.1->deeppavlov) (2.5.3)\n",
            "Collecting idna<2.8,>=2.5 (from requests==2.19.1->deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl (58kB)\n",
            "\u001b[K     |████████████████████████████████| 61kB 20.0MB/s \n",
            "\u001b[?25hCollecting urllib3<1.24,>=1.21.1 (from requests==2.19.1->deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/bd/c9/6fdd990019071a4a32a5e7cb78a1d92c53851ef4f56f62a3486e6a7d8ffb/urllib3-1.23-py2.py3-none-any.whl (133kB)\n",
            "\u001b[K     |████████████████████████████████| 143kB 41.2MB/s \n",
            "\u001b[?25hRequirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests==2.19.1->deeppavlov) (3.0.4)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests==2.19.1->deeppavlov) (2019.6.16)\n",
            "Requirement already satisfied: jsonschema>=2.5.1 in /usr/local/lib/python3.6/dist-packages (from flasgger==0.9.1->deeppavlov) (2.6.0)\n",
            "Requirement already satisfied: mistune in /usr/local/lib/python3.6/dist-packages (from flasgger==0.9.1->deeppavlov) (0.8.4)\n",
            "Requirement already satisfied: docopt>=0.6 in /usr/local/lib/python3.6/dist-packages (from pymorphy2==0.8->deeppavlov) (0.6.2)\n",
            "Collecting dawg-python>=0.7 (from pymorphy2==0.8->deeppavlov)\n",
            "  Downloading https://files.pythonhosted.org/packages/6a/84/ff1ce2071d4c650ec85745766c0047ccc3b5036f1d03559fd46bb38b5eeb/DAWG_Python-0.7.2-py2.py3-none-any.whl\n",
            "Collecting pymorphy2-dicts<3.0,>=2.4 (from pymorphy2==0.8->deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/02/51/2465fd4f72328ab50877b54777764d928da8cb15b74e2680fc1bd8cb3173/pymorphy2_dicts-2.4.393442.3710985-py2.py3-none-any.whl (7.1MB)\n",
            "\u001b[K     |████████████████████████████████| 7.1MB 37.2MB/s \n",
            "\u001b[?25hCollecting cryptography>=2.2.1 (from pyopenssl==18.0.0->deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/97/18/c6557f63a6abde34707196fb2cad1c6dc0dbff25a200d5044922496668a4/cryptography-2.7-cp34-abi3-manylinux1_x86_64.whl (2.3MB)\n",
            "\u001b[K     |████████████████████████████████| 2.3MB 28.5MB/s \n",
            "\u001b[?25hRequirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.6/dist-packages (from Jinja2>=2.10->flask==1.0.2->deeppavlov) (1.1.1)\n",
            "Requirement already satisfied: cffi!=1.11.3,>=1.8 in /usr/local/lib/python3.6/dist-packages (from cryptography>=2.2.1->pyopenssl==18.0.0->deeppavlov) (1.12.3)\n",
            "Collecting asn1crypto>=0.21.0 (from cryptography>=2.2.1->pyopenssl==18.0.0->deeppavlov)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl (101kB)\n",
            "\u001b[K     |████████████████████████████████| 102kB 26.9MB/s \n",
            "\u001b[?25hRequirement already satisfied: pycparser in /usr/local/lib/python3.6/dist-packages (from cffi!=1.11.3,>=1.8->cryptography>=2.2.1->pyopenssl==18.0.0->deeppavlov) (2.19)\n",
            "Building wheels for collected packages: pytelegrambotapi, overrides\n",
            "  Building wheel for pytelegrambotapi (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Stored in directory: /root/.cache/pip/wheels/b0/a0/fa/c3539fd47aa9f834230d64039c4bc620463bc7afc39b0f3f35\n",
            "  Building wheel for overrides (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Stored in directory: /root/.cache/pip/wheels/8d/52/86/e5a83b1797e7d263b458d2334edd2704c78508b3eea9323718\n",
            "Successfully built pytelegrambotapi overrides\n",
            "\u001b[31mERROR: yellowbrick 0.9.1 has requirement scikit-learn>=0.20, but you'll have scikit-learn 0.19.1 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: tensorflow 1.14.0 has requirement keras-applications>=1.0.6, but you'll have keras-applications 1.0.2 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: tensorflow 1.14.0 has requirement keras-preprocessing>=1.0.5, but you'll have keras-preprocessing 1.0.1 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: spacy 2.1.4 has requirement numpy>=1.15.0, but you'll have numpy 1.14.5 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: plotnine 0.5.1 has requirement pandas>=0.23.4, but you'll have pandas 0.23.1 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: mizani 0.5.4 has requirement pandas>=0.23.4, but you'll have pandas 0.23.1 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: imgaug 0.2.9 has requirement numpy>=1.15.0, but you'll have numpy 1.14.5 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: imbalanced-learn 0.4.3 has requirement scikit-learn>=0.20, but you'll have scikit-learn 0.19.1 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: google-colab 1.0.0 has requirement pandas~=0.24.0, but you'll have pandas 0.23.1 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: google-colab 1.0.0 has requirement requests~=2.21.0, but you'll have requests 2.19.1 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: fbprophet 0.5 has requirement pandas>=0.23.4, but you'll have pandas 0.23.1 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: fastai 1.0.54 has requirement numpy>=1.15, but you'll have numpy 1.14.5 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: datascience 0.10.6 has requirement folium==0.2.1, but you'll have folium 0.8.3 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: blis 0.2.4 has requirement numpy>=1.15.0, but you'll have numpy 1.14.5 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: albumentations 0.1.12 has requirement imgaug<0.2.7,>=0.2.5, but you'll have imgaug 0.2.9 which is incompatible.\u001b[0m\n",
            "Installing collected packages: Cython, numpy, idna, urllib3, requests, pytelegrambotapi, scipy, keras-preprocessing, keras-applications, keras, flask, pymorphy2-dicts-ru, scikit-learn, pandas, fuzzywuzzy, flask-cors, tqdm, flasgger, overrides, dawg-python, pymorphy2-dicts, pymorphy2, rusenttokenize, asn1crypto, cryptography, pyopenssl, deeppavlov\n",
            "  Found existing installation: Cython 0.29.10\n",
            "    Uninstalling Cython-0.29.10:\n",
            "      Successfully uninstalled Cython-0.29.10\n",
            "  Found existing installation: numpy 1.16.4\n",
            "    Uninstalling numpy-1.16.4:\n",
            "      Successfully uninstalled numpy-1.16.4\n",
            "  Found existing installation: idna 2.8\n",
            "    Uninstalling idna-2.8:\n",
            "      Successfully uninstalled idna-2.8\n",
            "  Found existing installation: urllib3 1.24.3\n",
            "    Uninstalling urllib3-1.24.3:\n",
            "      Successfully uninstalled urllib3-1.24.3\n",
            "  Found existing installation: requests 2.21.0\n",
            "    Uninstalling requests-2.21.0:\n",
            "      Successfully uninstalled requests-2.21.0\n",
            "  Found existing installation: scipy 1.3.0\n",
            "    Uninstalling scipy-1.3.0:\n",
            "      Successfully uninstalled scipy-1.3.0\n",
            "  Found existing installation: Keras-Preprocessing 1.1.0\n",
            "    Uninstalling Keras-Preprocessing-1.1.0:\n",
            "      Successfully uninstalled Keras-Preprocessing-1.1.0\n",
            "  Found existing installation: Keras-Applications 1.0.8\n",
            "    Uninstalling Keras-Applications-1.0.8:\n",
            "      Successfully uninstalled Keras-Applications-1.0.8\n",
            "  Found existing installation: Keras 2.2.4\n",
            "    Uninstalling Keras-2.2.4:\n",
            "      Successfully uninstalled Keras-2.2.4\n",
            "  Found existing installation: Flask 1.0.3\n",
            "    Uninstalling Flask-1.0.3:\n",
            "      Successfully uninstalled Flask-1.0.3\n",
            "  Found existing installation: scikit-learn 0.21.2\n",
            "    Uninstalling scikit-learn-0.21.2:\n",
            "      Successfully uninstalled scikit-learn-0.21.2\n",
            "  Found existing installation: pandas 0.24.2\n",
            "    Uninstalling pandas-0.24.2:\n",
            "      Successfully uninstalled pandas-0.24.2\n",
            "  Found existing installation: tqdm 4.28.1\n",
            "    Uninstalling tqdm-4.28.1:\n",
            "      Successfully uninstalled tqdm-4.28.1\n",
            "Successfully installed Cython-0.28.5 asn1crypto-0.24.0 cryptography-2.7 dawg-python-0.7.2 deeppavlov-0.4.0 flasgger-0.9.1 flask-1.0.2 flask-cors-3.0.6 fuzzywuzzy-0.16.0 idna-2.7 keras-2.2.0 keras-applications-1.0.2 keras-preprocessing-1.0.1 numpy-1.14.5 overrides-1.9 pandas-0.23.1 pymorphy2-0.8 pymorphy2-dicts-2.4.393442.3710985 pymorphy2-dicts-ru-2.4.404381.4453942 pyopenssl-18.0.0 pytelegrambotapi-3.5.2 requests-2.19.1 rusenttokenize-0.0.4 scikit-learn-0.19.1 scipy-1.1.0 tqdm-4.23.4 urllib3-1.23\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "idna",
                  "numpy",
                  "pandas",
                  "requests",
                  "tqdm",
                  "urllib3"
                ]
              }
            }
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "8foXOlPocHJv",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 978
        },
        "outputId": "1f7d58ca-10b9-4d6b-e2f3-84b02832e79e"
      },
      "source": [
        "!python -m deeppavlov install intents_snips"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2019-07-01 02:28:36.323 INFO in 'deeppavlov.core.common.file'['file'] at line 30: Interpreting 'intents_snips' as '/usr/local/lib/python3.6/dist-packages/deeppavlov/configs/classifiers/intents_snips.json'\n",
            "Collecting tensorflow==1.10.0\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/ee/e6/a6d371306c23c2b01cd2cb38909673d17ddd388d9e4b3c0f6602bfd972c8/tensorflow-1.10.0-cp36-cp36m-manylinux1_x86_64.whl (58.4MB)\n",
            "\u001b[K     |████████████████████████████████| 58.4MB 2.5MB/s \n",
            "\u001b[?25hRequirement already satisfied: absl-py>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.10.0) (0.7.1)\n",
            "Collecting setuptools<=39.1.0 (from tensorflow==1.10.0)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/8c/10/79282747f9169f21c053c562a0baa21815a8c7879be97abd930dbcf862e8/setuptools-39.1.0-py2.py3-none-any.whl (566kB)\n",
            "\u001b[K     |████████████████████████████████| 573kB 38.8MB/s \n",
            "\u001b[?25hRequirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.10.0) (0.8.0)\n",
            "Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.10.0) (0.33.4)\n",
            "Requirement already satisfied: protobuf>=3.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.10.0) (3.7.1)\n",
            "Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.10.0) (1.15.0)\n",
            "Collecting tensorboard<1.11.0,>=1.10.0 (from tensorflow==1.10.0)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/c6/17/ecd918a004f297955c30b4fffbea100b1606c225dbf0443264012773c3ff/tensorboard-1.10.0-py3-none-any.whl (3.3MB)\n",
            "\u001b[K     |████████████████████████████████| 3.3MB 38.0MB/s \n",
            "\u001b[?25hRequirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.10.0) (1.1.0)\n",
            "Requirement already satisfied: gast>=0.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.10.0) (0.2.2)\n",
            "Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.10.0) (1.12.0)\n",
            "Requirement already satisfied: numpy<=1.14.5,>=1.13.3 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.10.0) (1.14.5)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.11.0,>=1.10.0->tensorflow==1.10.0) (3.1.1)\n",
            "Requirement already satisfied: werkzeug>=0.11.10 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.11.0,>=1.10.0->tensorflow==1.10.0) (0.15.4)\n",
            "\u001b[31mERROR: magenta 0.3.19 has requirement tensorflow>=1.12.0, but you'll have tensorflow 1.10.0 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: google-colab 1.0.0 has requirement pandas~=0.24.0, but you'll have pandas 0.23.1 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: google-colab 1.0.0 has requirement requests~=2.21.0, but you'll have requests 2.19.1 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: datascience 0.10.6 has requirement folium==0.2.1, but you'll have folium 0.8.3 which is incompatible.\u001b[0m\n",
            "Installing collected packages: setuptools, tensorboard, tensorflow\n",
            "  Found existing installation: setuptools 41.0.1\n",
            "    Uninstalling setuptools-41.0.1:\n",
            "      Successfully uninstalled setuptools-41.0.1\n",
            "  Found existing installation: tensorboard 1.14.0\n",
            "    Uninstalling tensorboard-1.14.0:\n",
            "      Successfully uninstalled tensorboard-1.14.0\n",
            "  Found existing installation: tensorflow 1.14.0\n",
            "    Uninstalling tensorflow-1.14.0:\n",
            "      Successfully uninstalled tensorflow-1.14.0\n",
            "Successfully installed setuptools-39.1.0 tensorboard-1.10.0 tensorflow-1.10.0\n",
            "Collecting pybind11==2.2.3\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/12/90/0f92a575dc60c8fba6d0c91d6b45abdb1058da9ebed40400cbcfad2ac0a7/pybind11-2.2.3-py2.py3-none-any.whl (144kB)\n",
            "\u001b[K     |████████████████████████████████| 153kB 2.7MB/s \n",
            "\u001b[?25hInstalling collected packages: pybind11\n",
            "Successfully installed pybind11-2.2.3\n",
            "Collecting fastText==0.8.22 from git+https://github.com/deepmipt/fastText.git#egg=fastText==0.8.22\n",
            "  Cloning https://github.com/deepmipt/fastText.git to /tmp/pip-install-av051h01/fastText\n",
            "  Running command git clone -q https://github.com/deepmipt/fastText.git /tmp/pip-install-av051h01/fastText\n",
            "Requirement already satisfied: pybind11>=2.2 in /usr/local/lib/python3.6/dist-packages (from fastText==0.8.22) (2.2.3)\n",
            "Requirement already satisfied: setuptools>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from fastText==0.8.22) (39.1.0)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from fastText==0.8.22) (1.14.5)\n",
            "Building wheels for collected packages: fastText\n",
            "  Building wheel for fastText (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Stored in directory: /tmp/pip-ephem-wheel-cache-2y67puht/wheels/19/a8/8d/f2d5f95a573a8bbf7e818c1914e80d72899233bfb111c04539\n",
            "Successfully built fastText\n",
            "Installing collected packages: fastText\n",
            "Successfully installed fastText-0.8.22\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rBjOVuFVE1P4",
        "colab_type": "text"
      },
      "source": [
        "## Dataset downloading."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pRMjDk79E1Xf",
        "colab_type": "text"
      },
      "source": [
        "This tutorial uses dataset Stanford Sentiment Treebank (SST) from [paper](https://nlp.stanford.edu/~socherr/EMNLP2013_RNTN.pdf).\n",
        "\n",
        "The dataset contains unlabelled sentences divided to train/dev/test sets, phrases labelled with float sentiment value. Most of the sentences are contained in labelled list of phrases. Therefore, we are going to extract sentences coinciding with labelled phrases, convert their float sentiment to fine-grained (5 classes: very negative, negative, neutral, positive, very positive) and binary classes (negative and positive only), build two classifiers."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X2CSOOvQE6wA",
        "colab_type": "text"
      },
      "source": [
        "Let's download and extract the SST dataset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9evo51p-E6_u",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 72
        },
        "outputId": "707d48e3-ac91-4fb4-8c34-2665edf35327"
      },
      "source": [
        "from deeppavlov.core.data.utils import download\n",
        "\n",
        "download(\"./stanfordSentimentTreebank.zip\", source_url=\"http://files.deeppavlov.ai/datasets/stanfordSentimentTreebank.zip\")"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2019-07-01 02:30:05.291 INFO in 'deeppavlov.core.data.utils'['utils'] at line 63: Downloading from http://files.deeppavlov.ai/datasets/stanfordSentimentTreebank.zip to /content/stanfordSentimentTreebank.zip\n",
            "100%|██████████| 7.26M/7.26M [00:01<00:00, 3.83MB/s]\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eGe8JOi6FBBS",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 443
        },
        "outputId": "16c28985-9ffc-43f9-d330-ea63d237f12d"
      },
      "source": [
        "!unzip stanfordSentimentTreebank.zip"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Archive:  stanfordSentimentTreebank.zip\n",
            "   creating: stanfordSentimentTreebank/\n",
            "  inflating: stanfordSentimentTreebank/datasetSentences.txt  \n",
            "   creating: __MACOSX/\n",
            "   creating: __MACOSX/stanfordSentimentTreebank/\n",
            "  inflating: __MACOSX/stanfordSentimentTreebank/._datasetSentences.txt  \n",
            "  inflating: stanfordSentimentTreebank/datasetSplit.txt  \n",
            "  inflating: __MACOSX/stanfordSentimentTreebank/._datasetSplit.txt  \n",
            "  inflating: stanfordSentimentTreebank/dictionary.txt  \n",
            "  inflating: __MACOSX/stanfordSentimentTreebank/._dictionary.txt  \n",
            "  inflating: stanfordSentimentTreebank/original_rt_snippets.txt  \n",
            "  inflating: __MACOSX/stanfordSentimentTreebank/._original_rt_snippets.txt  \n",
            "  inflating: stanfordSentimentTreebank/README.txt  \n",
            "  inflating: __MACOSX/stanfordSentimentTreebank/._README.txt  \n",
            "  inflating: stanfordSentimentTreebank/sentiment_labels.txt  \n",
            "  inflating: __MACOSX/stanfordSentimentTreebank/._sentiment_labels.txt  \n",
            "  inflating: stanfordSentimentTreebank/SOStr.txt  \n",
            "  inflating: stanfordSentimentTreebank/STree.txt  \n",
            "  inflating: stanfordSentimentTreebank/test_binary.csv  \n",
            "  inflating: stanfordSentimentTreebank/test_fine_grained.csv  \n",
            "  inflating: stanfordSentimentTreebank/train_binary.csv  \n",
            "  inflating: stanfordSentimentTreebank/train_fine_grained.csv  \n",
            "  inflating: stanfordSentimentTreebank/valid_binary.csv  \n",
            "  inflating: stanfordSentimentTreebank/valid_fine_grained.csv  \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JhY2yQcoFEb0",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 105
        },
        "outputId": "1ed704fb-0028-4594-eaec-18dcd20b246d"
      },
      "source": [
        "!ls stanfordSentimentTreebank/"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "datasetSentences.txt\t  sentiment_labels.txt\t train_binary.csv\n",
            "datasetSplit.txt\t  SOStr.txt\t\t train_fine_grained.csv\n",
            "dictionary.txt\t\t  STree.txt\t\t valid_binary.csv\n",
            "original_rt_snippets.txt  test_binary.csv\t valid_fine_grained.csv\n",
            "README.txt\t\t  test_fine_grained.csv\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "-3ecyN0XcHJw"
      },
      "source": [
        "## Dataset Reader\n",
        "\n",
        "DatasetReaders are components for reading datasets from files. DeepPavlov contains several different DatasetReaders, one can use either presented DatasetReader or build his own component. \n",
        "\n",
        "The only requirements is the output of **DatasetReader**: \n",
        "* output must be a dictionary with three fields \"train\", \"valid\" and \"test\", \n",
        "* each dictionary value must be a list of corresponding samples,\n",
        "* each sample must be a tuple (x, y) where either x, y or both can also be lists of several inputs.\n",
        "\n",
        "**DOCS:** http://docs.deeppavlov.ai/en/latest/apiref/dataset_readers.html"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "sSgPZbPAcHJx",
        "colab": {}
      },
      "source": [
        "from deeppavlov.dataset_readers.basic_classification_reader import BasicClassificationDatasetReader"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "7IMfseBmcHJy",
        "scrolled": true,
        "colab": {}
      },
      "source": [
        "reader = BasicClassificationDatasetReader()\n",
        "data = reader.read(data_path=\"./stanfordSentimentTreebank\", \n",
        "                   train=\"train_binary.csv\", valid=\"valid_binary.csv\", test=\"test_binary.csv\",\n",
        "                   x=\"text\", y=\"binary_label\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "yHMgaOdfcHJz",
        "outputId": "91829aee-0ed7-4754-ec47-7bc5539a7460",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "data.keys()"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "dict_keys(['train', 'valid', 'test'])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Si7P1UU6cHJ1"
      },
      "source": [
        "For every samples we store label(s) as list because we don't know whether it is binary, multi-class or multi-label classification."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "iMTGWqyJcHJ2",
        "outputId": "d792024d-078b-4b13-dfa8-1079adaeac18",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 72
        }
      },
      "source": [
        "data[\"train\"][0]"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(\"The Rock is destined to be the 21st Century 's new `` Conan '' and that he 's going to make a splash even greater than Arnold Schwarzenegger , Jean-Claud Van Damme or Steven Segal .\",\n",
              " ['positive'])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "z4o1sPsrcHJ3"
      },
      "source": [
        "## Dataset Iterator\n",
        "\n",
        "DatasetIterators are components for iterating over datasets. DeepPavlov contains several different DatasetIterators, one can either use presented iterator or build his own component.\n",
        "\n",
        "DatasetIterator must have the following methods:\n",
        "* **gen_batches** - method generates batches of inputs and expected output to train neural networks. Output is a tuple of a batch of inputs and a batch of expected outputs.\n",
        "* **get_instances** - method gets all data for a selected data type (\"train\", \"valid\", \"test\"). Output is a tuple of all inputs for a data type and all expected outputs for a data type.\n",
        "* **split** - method merges/splits data of a selected data type from DatasetReader (\"train\", \"valid\", \"test\").\n",
        "\n",
        "**DOCS:** http://docs.deeppavlov.ai/en/latest/apiref/dataset_iterators.html"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "G1iWNhFmcHJ4",
        "colab": {}
      },
      "source": [
        "from deeppavlov.dataset_iterators.basic_classification_iterator import BasicClassificationDatasetIterator"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "7LtY4xKMcHJ5",
        "colab": {}
      },
      "source": [
        "iterator = BasicClassificationDatasetIterator(data, seed=42, shuffle=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "enx-deI6Fc81",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        },
        "outputId": "89fb5012-9915-4706-db15-dbb78cf27722"
      },
      "source": [
        "for batch in iterator.gen_batches(data_type=\"train\", batch_size=13):\n",
        "  print(batch)\n",
        "  break"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(('Kids should have a stirring time at this beautifully drawn movie .', 'It is quite a vision .', \"It 's rare to find a film to which the adjective ` gentle ' applies , but the word perfectly describes Pauline & Paulette .\", \"Despite terrific special effects and funnier gags , Harry Potter and the Chamber of Secrets finds a way to make J.K. Rowling 's marvelous series into a deadly bore .\", 'A harrowing account of a psychological breakdown .', \"If you 're a fan of the series you 'll love it and probably want to see it twice .\", \"An escapist confection that 's pure entertainment .\", \"It 's mostly a pleasure to watch .\", 'All three women deliver remarkable performances .', 'The IMAX screen enhances the personal touch of manual animation .', \"With amazing finesse , the film shadows Heidi 's trip back to Vietnam and the city where her mother , Mai Thi Kim , still lives .\", 'For those of an indulgent , slightly sunbaked and summery mind , Sex and Lucia may well prove diverting enough .', 'No laughs .'), (['positive'], ['positive'], ['positive'], ['negative'], ['positive'], ['positive'], ['positive'], ['positive'], ['positive'], ['positive'], ['positive'], ['positive'], ['negative']))\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "C7q-San0cHJ6"
      },
      "source": [
        "## Preprocessor\n",
        "\n",
        "We can preprocess text according to our needs. \n",
        "Let's define the most simple preprocessor - lower-casing.\n",
        "\n",
        "**DOCS:** http://docs.deeppavlov.ai/en/latest/apiref/models/preprocessors.html"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "wHzWodEgcHJ7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 176
        },
        "outputId": "ef91fe58-6b27-493d-a60d-18089511ea10"
      },
      "source": [
        "from deeppavlov.models.preprocessors.str_lower import StrLower"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Unzipping tokenizers/punkt.zip.\n",
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n",
            "[nltk_data] Downloading package perluniprops to /root/nltk_data...\n",
            "[nltk_data]   Unzipping misc/perluniprops.zip.\n",
            "[nltk_data] Downloading package nonbreaking_prefixes to\n",
            "[nltk_data]     /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/nonbreaking_prefixes.zip.\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "iP0hUsI5cHJ9",
        "colab": {}
      },
      "source": [
        "preprocessor = StrLower()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "bQt4cfRzcHJ_",
        "outputId": "f38233f4-9e6f-465d-d7d7-a24017dcfe5c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "preprocessor([\"The Rock is destined to be the 21st Century 's new `` Conan ''.\"])"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[\"the rock is destined to be the 21st century 's new `` conan ''.\"]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "t3vA74rscHKC"
      },
      "source": [
        "## Tokenizer\n",
        "\n",
        "We need to tokenize our texts because we are going to use word embeddings.\n",
        "DeepPavlov contains several different tokenizers, one can choose the most appropriate.\n",
        "\n",
        "**DOCS:** http://docs.deeppavlov.ai/en/latest/apiref/models/tokenizers.html"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "EcrmkaFZcHKD",
        "colab": {}
      },
      "source": [
        "from deeppavlov.models.tokenizers.nltk_tokenizer import NLTKTokenizer"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "b_HxUZoacHKE",
        "colab": {}
      },
      "source": [
        "tokenizer = NLTKTokenizer()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "DQmVPhS2cHKF",
        "outputId": "a308ce69-b20a-4894-bce9-b43be5049cdc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 283
        }
      },
      "source": [
        "tokenizer([\"The Rock is destined to be the 21st Century 's new `` Conan ''.\"])"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[['The',\n",
              "  'Rock',\n",
              "  'is',\n",
              "  'destined',\n",
              "  'to',\n",
              "  'be',\n",
              "  'the',\n",
              "  '21st',\n",
              "  'Century',\n",
              "  \"'\",\n",
              "  's',\n",
              "  'new',\n",
              "  '``',\n",
              "  'Conan',\n",
              "  \"''.\"]]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "ZqCHoa0YcHKH"
      },
      "source": [
        "## Embedder\n",
        "\n",
        "We are planning to use non-trainable GloVe word embeddings. Let's download file.\n",
        "\n",
        "**DOCS:** http://docs.deeppavlov.ai/en/latest/apiref/models/embedders.html\n",
        "\n",
        "Now we need to download GloVe embeddings file. One can download from [here](https://nlp.stanford.edu/projects/glove/) but it downloads more than 800 Mb. To save your time, you can download GloVe embeddings file from DeepPavlov (downloads 350 Mb)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "8BDAP9GVcHKI",
        "outputId": "7d02a4f0-a103-41d1-8e74-d4261b1c82d3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 72
        }
      },
      "source": [
        "from deeppavlov.core.data.utils import download\n",
        "\n",
        "download(\"./glove.6B.100d.txt\", source_url=\"http://files.deeppavlov.ai/embeddings/glove.6B.100d.txt\")"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2019-07-01 02:31:12.602 INFO in 'deeppavlov.core.data.utils'['utils'] at line 63: Downloading from http://files.deeppavlov.ai/embeddings/glove.6B.100d.txt to /content/glove.6B.100d.txt\n",
            "347MB [00:24, 14.1MB/s]\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "g9UH7BfxcHKJ"
      },
      "source": [
        "Now we can define GloVeEmbedder. Parameter `pad_zero` which is set to `True` determines whether to pad embedded batch of tokens to the longest sample length."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "SP0yUvrXcHKK",
        "outputId": "663ec73e-59a0-4085-f42a-71088901a9e3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 90
        }
      },
      "source": [
        "from deeppavlov.models.embedders.glove_embedder import GloVeEmbedder\n",
        "\n",
        "embedder = GloVeEmbedder(load_path=\"./glove.6B.100d.txt\", \n",
        "                         pad_zero=True  # means whether to pad up to the longest sample in a batch\n",
        "                        )"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2019-07-01 02:31:37.943 INFO in 'deeppavlov.models.embedders.glove_embedder'['glove_embedder'] at line 52: [loading GloVe embeddings from `/content/glove.6B.100d.txt`]\n",
            "/usr/local/lib/python3.6/dist-packages/smart_open/smart_open_lib.py:398: UserWarning: This function is deprecated, use smart_open.open instead. See the migration notes for details: https://github.com/RaRe-Technologies/smart_open/blob/master/README.rst#migrating-to-the-new-open-function\n",
            "  'See the migration notes for details: %s' % _MIGRATION_NOTES_URL\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "539oiSQ4YHZV",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "a53288fd-f8ee-4773-eb85-29fe58abee9b"
      },
      "source": [
        "embedder.dim"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "100"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "km2NCO5McHKL",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 496
        },
        "outputId": "c1b48ff2-8255-4e14-c057-810c7443d4ca"
      },
      "source": [
        "embedder(tokenizer(preprocessor([\"The Rock is destined to be the 21st Century 's new 'Conan'.\",\n",
        "                                 \"The Rock is a new 'Conan'.\"])))"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[[-0.038194, -0.24487 ,  0.72812 , ..., -0.1459  ,  0.8278  ,\n",
              "          0.27062 ],\n",
              "        [-0.68387 ,  0.39176 ,  0.5367  , ..., -0.14145 ,  1.3115  ,\n",
              "          0.31476 ],\n",
              "        [-0.54264 ,  0.41476 ,  1.0322  , ..., -1.2969  ,  0.76217 ,\n",
              "          0.46349 ],\n",
              "        ...,\n",
              "        [-0.34562 , -0.24993 ,  0.58678 , ..., -1.3106  ,  1.0294  ,\n",
              "         -0.058794],\n",
              "        [-0.7287  , -0.40513 ,  0.25123 , ..., -0.78115 , -0.45564 ,\n",
              "          0.10672 ],\n",
              "        [ 0.      ,  0.      ,  0.      , ...,  0.      ,  0.      ,\n",
              "          0.      ]],\n",
              "\n",
              "       [[-0.038194, -0.24487 ,  0.72812 , ..., -0.1459  ,  0.8278  ,\n",
              "          0.27062 ],\n",
              "        [-0.68387 ,  0.39176 ,  0.5367  , ..., -0.14145 ,  1.3115  ,\n",
              "          0.31476 ],\n",
              "        [-0.54264 ,  0.41476 ,  1.0322  , ..., -1.2969  ,  0.76217 ,\n",
              "          0.46349 ],\n",
              "        ...,\n",
              "        [ 0.      ,  0.      ,  0.      , ...,  0.      ,  0.      ,\n",
              "          0.      ],\n",
              "        [ 0.      ,  0.      ,  0.      , ...,  0.      ,  0.      ,\n",
              "          0.      ],\n",
              "        [ 0.      ,  0.      ,  0.      , ...,  0.      ,  0.      ,\n",
              "          0.      ]]], dtype=float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "pqqIo_6FcHKN",
        "outputId": "f98042b8-3e52-4e64-8f84-5abf4ac59f20",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "embedder(tokenizer(preprocessor([\"The Rock is destined to be the 21st Century 's new 'Conan'.\",\n",
        "                                 \"The Rock is a new 'Conan'.\"]))).shape"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(2, 15, 100)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "fdyZP9obcHKO"
      },
      "source": [
        "## Vocabulary of classes\n",
        "\n",
        "By default, we assume that we have different classes which also can be given as strings. Therefore, we need to convert them to something more appropriate for classifier. For example, neural classifiers always need to get **one-hot** representation of classes. To get one-hot representation we have to collect a dictionary with all the classes appeared (if needed one can add \"unknown\" class), index class samples and convert to one-hot representation.\n",
        "\n",
        "**DOCS:** http://docs.deeppavlov.ai/en/latest/apiref/core/data.html"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "3wZMh_6VcHKO",
        "colab": {}
      },
      "source": [
        "from deeppavlov.core.data.simple_vocab import SimpleVocabulary"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "_7O0c273cHKQ",
        "outputId": "d40948d1-c244-4944-f256-cabb8fbd309f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        }
      },
      "source": [
        "vocab = SimpleVocabulary(save_path=\"./binary_classes.dict\")"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2019-07-01 02:32:49.358 WARNING in 'deeppavlov.core.models.serializable'['serializable'] at line 47: No load path is set for SimpleVocabulary in 'infer' mode. Using save path instead\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OpRDirSeXLav",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "e97e2e19-fc1a-4770-babd-d6d5d95fc1ee"
      },
      "source": [
        "iterator.get_instances(data_type=\"train\")"
      ],
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "((\"The Rock is destined to be the 21st Century 's new `` Conan '' and that he 's going to make a splash even greater than Arnold Schwarzenegger , Jean-Claud Van Damme or Steven Segal .\",\n",
              "  \"The gorgeously elaborate continuation of `` The Lord of the Rings '' trilogy is so huge that a column of words can not adequately describe co-writer\\\\/director Peter Jackson 's expanded vision of J.R.R. Tolkien 's Middle-earth .\",\n",
              "  'Singer\\\\/composer Bryan Adams contributes a slew of songs -- a few potential hits , a few more simply intrusive to the story -- but the whole package certainly captures the intended , er , spirit of the piece .',\n",
              "  'Yet the act is still charming here .',\n",
              "  \"Whether or not you 're enlightened by any of Derrida 's lectures on `` the other '' and `` the self , '' Derrida is an undeniably fascinating and playful fellow .\",\n",
              "  'Just the labour involved in creating the layered richness of the imagery in this chiaroscuro of madness and light is astonishing .',\n",
              "  'Part of the charm of Satin Rouge is that it avoids the obvious with humour and lightness .',\n",
              "  \"a screenplay more ingeniously constructed than `` Memento ''\",\n",
              "  \"`` Extreme Ops '' exceeds expectations .\",\n",
              "  'Good fun , good action , good acting , good dialogue , good pace , good cinematography .',\n",
              "  'Dramas like this make it human .',\n",
              "  'Still , this flick is fun , and host to some truly excellent sequences .',\n",
              "  'Australian actor\\\\/director John Polson and award-winning English cinematographer Giles Nuttgens make a terrific effort at disguising the obvious with energy and innovation .',\n",
              "  'You walk out of The Good Girl with mixed emotions -- disapproval of Justine combined with a tinge of understanding for her actions .',\n",
              "  'If you love reading and\\\\/or poetry , then by all means check it out .',\n",
              "  \"You 'll probably love it .\",\n",
              "  \"`` Frailty '' has been written so well , that even a simple `` Goddammit ! ''\",\n",
              "  \"Grenier is terrific , bringing an unforced , rapid-fire delivery to Toback 's Heidegger - and Nietzsche-referencing dialogue .\",\n",
              "  'The actors are fantastic .',\n",
              "  'They are what makes it worth the trip to the theatre .',\n",
              "  \"This is n't a new idea .\",\n",
              "  \"It 's been done before but never so vividly or with so much passion .\",\n",
              "  \"What `` Empire '' lacks in depth it makes up for with its heart .\",\n",
              "  'Light , silly , photographed with colour and depth , and rather a good time .',\n",
              "  \"But tongue-in-cheek preposterousness has always been part of For the most part Wilde 's droll whimsy helps `` Being Earnest '' overcome its weaknesses and Parker 's creative interference ...\",\n",
              "  \"Much of the movie 's charm lies in the utter cuteness of Stuart and Margolo .\",\n",
              "  'Their computer-animated faces are very expressive .',\n",
              "  'Much of it comes from the brave , uninhibited performances by its lead actors .',\n",
              "  \"`` Auto Focus '' works as an unusual biopic and document of male swingers in the Playboy era\",\n",
              "  \"... a sour little movie at its core ; an exploration of the emptiness that underlay the relentless gaiety of the 1920 's ... The film 's ending has a `` What was it all for ? ''\",\n",
              "  \"feeling to it , but like the 1920 's , the trip there is a great deal of fun .\",\n",
              "  'No. .',\n",
              "  \"`` Cremaster 3 '' should come with the warning `` For serious film buffs only ! ''\",\n",
              "  'Made me unintentionally famous -- as the queasy-stomached critic who staggered from the theater and blacked out in the lobby .',\n",
              "  \"But believe it or not , it 's one of the most beautiful , evocative works I 've seen .\",\n",
              "  '... a spoof comedy that carries its share of laughs -- sometimes a chuckle , sometimes a guffaw and , to my great pleasure , the occasional belly laugh .',\n",
              "  'The wanton slipperiness of \\\\* Corpus and its amiable jerking and reshaping of physical time and space would make it a great piece to watch with kids and use to introduce video as art .',\n",
              "  'For those who pride themselves on sophisticated , discerning taste , this might not seem like the proper cup of tea , however it is almost guaranteed that even the stuffiest cinema goers will laugh their \\\\*\\\\*\\\\* off for an hour-and-a-half .',\n",
              "  'It cuts to the core of what it actually means to face your fears , to be a girl in a world of boys , to be a boy truly in love with a girl , and to ride the big metaphorical wave that is life -- wherever it takes you .',\n",
              "  'A welcome relief from baseball movies that try too hard to be mythic , this one is a sweet and modest and ultimately winning story .',\n",
              "  'It has more than a few moments that are insightful enough to be fondly remembered in the endlessly challenging maze of moviegoing .',\n",
              "  \"An uncluttered , resonant gem that relays its universal points without lectures or confrontations . '\",\n",
              "  'A smart , steamy mix of road movie , coming-of-age story and political satire .',\n",
              "  'The modern-day royals have nothing on these guys when it comes to scandals .',\n",
              "  \"It 's only in fairy tales that princesses that are married for political reason live happily ever after .\",\n",
              "  'A terrific B movie -- in fact , the best in recent memory .',\n",
              "  \"`` Birthday Girl '' is an actor 's movie first and foremost .\",\n",
              "  'I walked away from this new version of E.T. just as I hoped I would -- with moist eyes .',\n",
              "  \"For devotees of French cinema , Safe Conduct is so rich with period minutiae it 's like dying and going to celluloid heaven .\",\n",
              "  \"What 's really so appealing about the characters is their resemblance to everyday children .\",\n",
              "  'An absurdist spider web .',\n",
              "  \"If you 're as happy listening to movies as you are watching them , and the slow parade of human frailty fascinates you , then you 're at the right film .\",\n",
              "  \"Mention `` Solaris '' five years from now and I 'm sure those who saw it will have an opinion to share .\",\n",
              "  \"Allen 's funniest and most likeable movie in years .\",\n",
              "  \"It 's a glorious spectacle like those D.W. Griffith made in the early days of silent film .\",\n",
              "  'This comic gem is as delightful as it is derivative .',\n",
              "  'More timely than its director could ever have dreamed , this quietly lyrical tale probes the ambiguous welcome extended by Iran to the Afghani refugees who streamed across its borders , desperate for work and food .',\n",
              "  'The leaping story line , shaped by director Peter Kosminsky into sharp slivers and cutting impressions , shows all the signs of rich detail condensed into a few evocative images and striking character traits .',\n",
              "  'The production has been made with an enormous amount of affection , so we believe these characters love each other .',\n",
              "  'Certainly the performances are worthwhile .',\n",
              "  'Winds up being both revelatory and narcissistic , achieving some honest insight into relationships that most high-concept films candy-coat with pat storylines , precious circumstances and beautiful stars .',\n",
              "  'Watching these eccentrics is both inspiring and pure joy .',\n",
              "  'Steven Spielberg brings us another masterpiece',\n",
              "  \"Finally , the French-produced `` Read My Lips '' is a movie that understands characters must come first .\",\n",
              "  'Ms. Seigner and Mr. Serrault bring fresh , unforced naturalism to their characters .',\n",
              "  'Allen shows he can outgag any of those young whippersnappers making moving pictures today .',\n",
              "  'A good film with a solid pedigree both in front of and , more specifically , behind the camera .',\n",
              "  'By no means a slam-dunk and sure to ultimately disappoint the action fans who will be moved to the edge of their seats by the dynamic first act , it still comes off as a touching , transcendent love story .',\n",
              "  \"I encourage young and old alike to go see this unique and entertaining twist on the classic whale 's tale -- you wo n't be sorry !\",\n",
              "  \"A literary detective story is still a detective story and aficionados of the whodunit wo n't be disappointed .\",\n",
              "  \"High Crimes steals so freely from other movies and combines enough disparate types of films that it ca n't help but engage an audience .\",\n",
              "  \"If you 're a fan of the series you 'll love it and probably want to see it twice .\",\n",
              "  'I will be .',\n",
              "  \"It celebrates the group 's playful spark of nonconformity , glancing vividly back at what Hibiscus grandly called his ` angels of light . '\",\n",
              "  'The story ... is inspiring , ironic , and revelatory of just how ridiculous and money-oriented the record industry really is .',\n",
              "  'It is also a testament to the integrity and vision of the band .',\n",
              "  'Laced with liberal doses of dark humor , gorgeous exterior photography , and a stable-full of solid performances , No Such Thing is a fascinating little tale .',\n",
              "  \"Huppert 's show to steal and she makes a meal of it , channeling Kathy Baker 's creepy turn as the repressed mother on Boston Public just as much as 8 Women 's Augustine .\",\n",
              "  \"Nair does n't treat the issues lightly .\",\n",
              "  'She allows each character to confront their problems openly and honestly .',\n",
              "  'One of the best silly horror movies of recent memory , with some real shocks in store for unwary viewers .',\n",
              "  'The work of a filmmaker who has secrets buried at the heart of his story and knows how to take time revealing them .',\n",
              "  'Strange occurrences build in the mind of the viewer and take on extreme urgency .',\n",
              "  'Has a certain ghoulish fascination , and generates a fair amount of B-movie excitement .',\n",
              "  'Familiar but utterly delightful .',\n",
              "  'A fascinating , dark thriller that keeps you hooked on the delicious pulpiness of its lurid fiction .',\n",
              "  'The film aims to be funny , uplifting and moving , sometimes all at once .',\n",
              "  'The extent to which it succeeds is impressive .',\n",
              "  'The film brilliantly shines on all the characters , as the direction is intelligently accomplished .',\n",
              "  'While not for every taste , this often very funny collegiate gross-out comedy goes a long way toward restoring the luster of the National Lampoon film franchise , too long reduced to direct-to-video irrelevancy .',\n",
              "  'As broad and cartoonish as the screenplay is , there is an accuracy of observation in the work of the director , Frank Novak , that keeps the film grounded in an undeniable social realism .',\n",
              "  \"In addition to Hoffman 's powerful acting clinic , this is that rare drama that offers a thoughtful and rewarding glimpse into the sort of heartache everyone has felt , or will feel someday .\",\n",
              "  \"Jeffrey Tambor 's performance as the intelligent jazz-playing exterminator is Oscar-worthy .\",\n",
              "  \"From the opening strains of the Average White Band 's `` Pick up the Pieces '' , you can feel the love .\",\n",
              "  \"Stevens ' vibrant creative instincts are the difference between this and countless other flicks about guys and dolls .\",\n",
              "  \"that it 'll probably be the best and most mature comedy of the 2002 summer season speaks more of the season than the picture\",\n",
              "  \"Meyjes ' provocative film might be called an example of the haphazardness of evil .\",\n",
              "  'The movie is pretty funny now and then without in any way demeaning its subjects .',\n",
              "  \"imagine a scenario where Bergman approaches Swedish fatalism using Gary Larson 's Far Side humor\",\n",
              "  'It is most remarkable not because of its epic scope , but because of the startling intimacy it achieves despite that breadth .',\n",
              "  \"It 's not a great monster movie .\",\n",
              "  \"Others , more attuned to the anarchist maxim that ` the urge to destroy is also a creative urge ' , or more willing to see with their own eyes , will find Morrison 's iconoclastic uses of technology to be liberating .\",\n",
              "  'Miller tells this very compelling tale with little fuss or noise , expertly plucking tension from quiet .',\n",
              "  'Time Out is existential drama without any of the pretension associated with the term .',\n",
              "  \"It 's a sweet , laugh-a-minute crowd pleaser that lifts your spirits as well as the corners of your mouth .\",\n",
              "  \"It 's tough to watch , but it 's a fantastic movie .\",\n",
              "  'The best animated feature to hit theaters since Beauty and the Beast 11 years ago .',\n",
              "  \"What saves this deeply affecting film from being merely a collection of wrenching cases is Corcuera 's attention to detail .\",\n",
              "  \"Pacino is the best he 's been in years and Keener is marvelous .\",\n",
              "  'A solid , spooky entertainment worthy of the price of a ticket .',\n",
              "  'By turns fanciful , grisly and engagingly quixotic .',\n",
              "  '... very funny , very enjoyable ...',\n",
              "  \"Adaptation is intricately constructed and in a strange way nails all of Orlean 's themes without being a true adaptation of her book .\",\n",
              "  \"So purely enjoyable that you might not even notice it 's a fairly straightforward remake of Hollywood comedies such as Father of the Bride .\",\n",
              "  'Moonlight Mile gives itself the freedom to feel contradictory things .',\n",
              "  'It is sentimental but feels free to offend , is analytical and then surrenders to the illogic of its characters , is about grief and yet permits laughter .',\n",
              "  \"The real triumphs in Igby come from Philippe , who makes Oliver far more interesting than the character 's lines would suggest , and Sarandon , who could n't be better as a cruel but weirdly likable WASP matron .\",\n",
              "  'Robin Williams has thankfully ditched the saccharine sentimentality of Bicentennial Man in favour of an altogether darker side .',\n",
              "  \"If you 're willing to have fun with it , you wo n't feel cheated by the high infidelity of Unfaithful .\",\n",
              "  'Australia : Land Beyond Time is an enjoyable Big Movie primarily because Australia is a weirdly beautiful place .',\n",
              "  \"Hoffman 's performance is authentic to the core of his being .\",\n",
              "  'Told just proficiently enough to trounce its overly comfortable trappings .',\n",
              "  \"An enthralling aesthetic experience , one that 's steeped in mystery and a ravishing , baroque beauty .\",\n",
              "  'The quirky drama touches the heart and the funnybone thanks to the energetic and always surprising performance by Rachel Griffiths .',\n",
              "  'A captivating coming-of-age story that may also be the first narrative film to be truly informed by the wireless age .',\n",
              "  \"What could have been a daytime soap opera is actually a compelling look at a young woman 's tragic odyssey .\",\n",
              "  'Duvall is strong as always .',\n",
              "  'A no-holds-barred cinematic treat .',\n",
              "  \"You 'd have to be a most hard-hearted person not to be moved by this drama .\",\n",
              "  \"Allen 's underestimated charm delivers more goodies than lumps of coal .\",\n",
              "  'Measured against practically any like-themed film other than its Oscar-sweeping franchise predecessor The Silence of the Lambs , Red Dragon rates as an exceptional thriller .',\n",
              "  'Maelstrom is strange and compelling , engrossing and different , a moral tale with a twisted sense of humor .',\n",
              "  'It makes you believe the cast and crew thoroughly enjoyed themselves and believed in their small-budget film .',\n",
              "  \"Too often , Son of the Bride becomes an exercise in trying to predict when a preordained `` big moment '' will occur and not `` if . ''\",\n",
              "  'The picture uses humor and a heartfelt conviction to tell a story about discovering your destination in life , but also acknowledging the places , and the people , from whence you came .',\n",
              "  'A terrific insider look at the star-making machinery of tinseltown .',\n",
              "  \"It 's a diverting enough hour-and-a-half for the family audience .\",\n",
              "  'A party-hearty teen flick that scalds like acid .',\n",
              "  'As giddy and whimsical and relevant today as it was 270 years ago .',\n",
              "  'The film offers an intriguing what-if premise .',\n",
              "  'The Pianist is the film Roman Polanski may have been born to make .',\n",
              "  'This version does justice both to Stevenson and to the sci-fi genre .',\n",
              "  'Poignant and delicately complex .',\n",
              "  'Enough may pander to our basest desires for payback , but unlike many revenge fantasies , it ultimately delivers .',\n",
              "  \"Cho 's latest comic set is n't as sharp or as fresh as I 'm the One That I Want ... but it 's still damn funny stuff .\",\n",
              "  'In The Pianist , Polanski is saying what he has long wanted to say , confronting the roots of his own preoccupations and obsessions , and he allows nothing to get in the way .',\n",
              "  \"Despite the film 's shortcomings , the stories are quietly moving .\",\n",
              "  'If you come from a family that eats , meddles , argues , laughs , kibbitzes and fights together , then go see this delightful comedy .',\n",
              "  'This bracingly truthful antidote to Hollywood teenage movies that slather Clearasil over the blemishes of youth captures the combustible mixture of a chafing inner loneliness and desperate grandiosity that tend to characterize puberty .',\n",
              "  \"It 's the filmmakers ' post-camp comprehension of what made old-time B movies good-bad that makes Eight Legged Freaks a perfectly entertaining summer diversion .\",\n",
              "  \"The film 's strength is n't in its details , but in the larger picture it paints - of a culture in conflict with itself , with the thin veneer of nationalism that covers our deepest , media-soaked fears .\",\n",
              "  'It has its faults , but it is a kind , unapologetic , sweetheart of a movie , and Mandy Moore leaves a positive impression .',\n",
              "  'The Saigon of 1952 is an uneasy mix of sensual delights and simmering violence , and The Quiet American brings us right into the center of that world .',\n",
              "  \"Despite its shortcomings , Girls Ca n't Swim represents an engaging and intimate first feature by a talented director to watch , and it 's a worthy entry in the French coming-of-age genre .\",\n",
              "  'With Dirty Deeds , David Caesar has stepped into the mainstream of filmmaking with an assurance worthy of international acclaim and with every cinematic tool well under his control -- driven by a natural sense for what works on screen .',\n",
              "  'The humor and humanity of Monsoon Wedding are in perfect balance .',\n",
              "  \"Lookin ' for sin , American-style ?\",\n",
              "  'A compelling motion picture that illustrates an American tragedy .',\n",
              "  'As comedic spotlights go , Notorious C.H.O. hits all the verbal marks it should .',\n",
              "  \"It 's a day at the beach -- with air conditioning and popcorn .\",\n",
              "  \"Frida is n't that much different from many a Hollywood romance .\",\n",
              "  \"What sets it apart is the vision that Taymor , the avant garde director of Broadway 's The Lion King and the film Titus , brings .\",\n",
              "  'Stevens has a flair for dialogue comedy , the film operates nicely off the element of surprise , and the large cast is solid .',\n",
              "  'Extremely well acted by the four primary actors , this is a seriously intended movie that is not easily forgotten .',\n",
              "  'The film exudes the urbane sweetness that Woody Allen seems to have bitterly forsaken .',\n",
              "  'K-19 : The Widowmaker is derivative , overlong , and bombastic -- yet surprisingly entertaining .',\n",
              "  \"It 's good , hard-edged stuff , violent and a bit exploitative but also nicely done , morally alert and street-smart .\",\n",
              "  \"Cineasts will revel in those visual in-jokes , as in the film 's verbal pokes at everything from the likes of Miramax chief Harvey Weinstein 's bluff personal style to the stylistic rigors of Denmark 's Dogma movement .\",\n",
              "  \"It 's a rare window on an artistic collaboration .\",\n",
              "  '... begins with promise , but runs aground after being snared in its own tangled plot .',\n",
              "  \"Perhaps the best sports movie I 've ever seen .\",\n",
              "  \"Cho 's timing is priceless .\",\n",
              "  'A sensitive , moving , brilliantly constructed work .',\n",
              "  'An edgy thriller that delivers a surprising punch .',\n",
              "  \"A reasonably entertaining sequel to 1994 's surprise family hit that may strain adult credibility .\",\n",
              "  'there is truth here',\n",
              "  \"a confident , richly acted , emotionally devastating piece of work and 2002 's first great film\",\n",
              "  'A touching , small-scale story of family responsibility and care in the community .',\n",
              "  'Arteta directs one of the best ensemble casts of the year',\n",
              "  \"The casting of von Sydow ... is itself Intacto 's luckiest stroke .\",\n",
              "  \"No , it 's not as single-minded as John Carpenter 's original , but it 's sure a lot smarter and more unnerving than the sequels .\",\n",
              "  'A gem of a romantic crime comedy that turns out to be clever , amusing and unpredictable .',\n",
              "  \"Stands as one of the year 's most intriguing movie experiences , letting its imagery speak for it while it forces you to ponder anew what a movie can be .\",\n",
              "  'This movie got me grinning .',\n",
              "  \"There 's a part of us that can not help be entertained by the sight of someone getting away with something .\",\n",
              "  \"An old-fashioned drama of substance about a teacher 's slide down the slippery slope of dishonesty after an encounter with the rich and the powerful who have nothing but disdain for virtue .\",\n",
              "  \"What 's not to like about a movie with a ` children 's ' song that includes the line ` My stepdad 's not mean , he 's just adjusting ' ?\",\n",
              "  'The pleasures that it does afford may be enough to keep many moviegoers occupied amidst some of the more serious-minded concerns of other year-end movies .',\n",
              "  'Not everyone will welcome or accept The Trials of Henry Kissinger as faithful portraiture , but few can argue that the debate it joins is a necessary and timely one .',\n",
              "  'There are no special effects , and no Hollywood endings .',\n",
              "  \"Like the original , this version is raised a few notches above kiddie fantasy pablum by Allen 's astringent wit .\",\n",
              "  \"Despite its Hawaiian setting , the science-fiction trimmings and some moments of rowdy slapstick , the basic plot of `` Lilo '' could have been pulled from a tear-stained vintage Shirley Temple script .\",\n",
              "  'Filled with honest performances and exceptional detail , Baran is a gentle film with dramatic punch , a haunting ode to humanity .',\n",
              "  \"In its dry and forceful way , it delivers the same message as Jiri Menzel 's Closely Watched Trains and Danis Tanovic 's No Man 's Land .\",\n",
              "  '... a triumph of emotionally and narratively complex filmmaking .',\n",
              "  \"Simply put , `` Far From Heaven '' is a masterpiece .\",\n",
              "  'An intense and effective film about loneliness and the chilly anonymity of the environments where so many of us spend so much of our time .',\n",
              "  \"Although fairly involving as far as it goes , the film does n't end up having much that is fresh to say about growing up Catholic or , really , anything .\",\n",
              "  'A very funny romantic comedy about two skittish New York middle-agers who stumble into a relationship and then struggle furiously with their fears and foibles .',\n",
              "  'Top-notch action powers this romantic drama .',\n",
              "  'Beresford nicely mixes in as much humor as pathos to take us on his sentimental journey of the heart .',\n",
              "  \"It really is a shame that more wo n't get an opportunity to embrace small , sweet ` Evelyn . '\",\n",
              "  'I stopped thinking about how good it all was , and started doing nothing but reacting to it - feeling a part of its grand locations , thinking urgently as the protagonists struggled , feeling at the mercy of its inventiveness , gasping at its visual delights .',\n",
              "  'Probably the best case for Christianity since Chesterton and Lewis .',\n",
              "  'A gently funny , sweetly adventurous film that makes you feel genuinely good , that is to say , entirely unconned by false sentiment or sharp , overmanipulative Hollywood practices .',\n",
              "  'Would be an unendurable viewing experience for this ultra-provincial New Yorker if 26-year-old Reese Witherspoon were not on hand to inject her pure fantasy character , Melanie Carmichael , with a massive infusion of old-fashioned Hollywood magic .',\n",
              "  'Visually fascinating ... an often intense character study about fathers and sons , loyalty and duty .',\n",
              "  'A lyrical metaphor for cultural and personal self-discovery and a picaresque view of a little-remembered world .',\n",
              "  'A slick , engrossing melodrama .',\n",
              "  \"S1M0NE 's satire is not subtle , but it is effective .\",\n",
              "  \"It 's a quirky , off-beat project .\",\n",
              "  'While some will object to the idea of a Vietnam picture with such a rah-rah , patriotic tone , Soldiers ultimately achieves its main strategic objective : dramatizing the human cost of the conflict that came to define a generation .',\n",
              "  \"Even if you do n't know the band or the album 's songs by heart , you will enjoy seeing how both evolve , and you will also learn a good deal about the state of the music business in the 21st Century .\",\n",
              "  'The solid filmmaking and convincing characters makes this a high water mark for this genre .',\n",
              "  'Films about loss , grief and recovery are pretty valuable these days .',\n",
              "  'Seen in that light , Moonlight Mile should strike a nerve in many .',\n",
              "  \"It 's endlessly inventive , consistently intelligent and sickeningly savage .\",\n",
              "  'It is definitely worth seeing .',\n",
              "  'An impeccable study in perversity .',\n",
              "  \"Far From Heaven is a dazzling conceptual feat , but more than that , it 's a work of enthralling drama .\",\n",
              "  'A movie that both thrills the eye and , in its over-the-top way , touches the heart .',\n",
              "  \"Stuffed to the brim with ideas , American instigator Michael Moore 's film is a rambling examination of American gun culture that uses his usual modus operandi of crucifixion through juxtaposition .\",\n",
              "  'Affectionately reminds us that , in any language , the huge stuff in life can usually be traced back to the little things .',\n",
              "  'The large-frame IMAX camera lends itself beautifully to filming the teeming life on the reefs , making this gorgeous film a must for everyone from junior scientists to grown-up fish lovers .',\n",
              "  'A story about intelligent high school students that deals with first love sweetly but also seriously .',\n",
              "  'It is also beautifully acted .',\n",
              "  \"It 's a technically superb film , shining with all the usual Spielberg flair , expertly utilizing the talents of his top-notch creative team .\",\n",
              "  'Wilco fans will have a great time , and the movie should win the band a few new converts , too .',\n",
              "  \"Tsai has a well-deserved reputation as one of the cinema world 's great visual stylists , and in this film , every shot enhances the excellent performances .\",\n",
              "  \"The fact is that the screen is most alive when it seems most likely that Broomfield 's interviewees , or even himself , will not be for much longer .\",\n",
              "  'Leguizamo and Jones are both excellent and the rest of the cast is uniformly superb .',\n",
              "  'I liked this film a lot ...',\n",
              "  \"... there is enough originality in ` Life ' to distance it from the pack of paint-by-number romantic comedies that so often end up on cinema screens .\",\n",
              "  'A solid and refined piece of moviemaking imbued with passion and attitude .',\n",
              "  'Nettelbeck has crafted an engaging fantasy of flavours and emotions , one part romance novel , one part recipe book .',\n",
              "  'With or without the sex , a wonderful tale of love and destiny , told well by a master storyteller',\n",
              "  \"On the surface a silly comedy , Scotland , PA would be forgettable if it were n't such a clever adaptation of the bard 's tragic play .\",\n",
              "  'A weird , arresting little ride .',\n",
              "  'A first-class road movie that proves you can run away from home , but your ego and all your problems go with you .',\n",
              "  \"You might want to take a reality check before you pay the full ticket price to see `` Simone , '' and consider a DVD rental instead .\",\n",
              "  'Well cast and well directed - a powerful drama with enough sardonic wit to keep it from being maudlin .',\n",
              "  'A backstage must-see for true fans of comedy .',\n",
              "  \"There 's back-stabbing , inter-racial desire and , most importantly , singing and dancing .\",\n",
              "  \"The film sounds like the stuff of lurid melodrama , but what makes it interesting as a character study is the fact that the story is told from Paul 's perspective .\",\n",
              "  'Jones ... makes a great impression as the writer-director of this little $ 1.8 million charmer , which may not be cutting-edge indie filmmaking but has a huge heart .',\n",
              "  'In the disturbingly involving family dysfunctional drama How I Killed My Father , French director Anne Fontaine delivers an inspired portrait of male-ridden angst and the emotional blockage that accompanies this human condition',\n",
              "  \"Below may not mark Mr. Twohy 's emergence into the mainstream , but his promise remains undiminished .\",\n",
              "  \"There 's no reason to miss Interview with the Assassin\",\n",
              "  \"Happily stays close to the ground in a spare and simple manner and does n't pummel us with phony imagery or music .\",\n",
              "  'Its sheer dynamism is infectious .',\n",
              "  'For his first attempt at film noir , Spielberg presents a fascinating but flawed look at the near future .',\n",
              "  \"Perhaps it 's cliche to call the film ` refreshing , ' but it is .\",\n",
              "  \"` Drumline ' shows a level of young , Black manhood that is funny , touching , smart and complicated .\",\n",
              "  'It does give a taste of the Burning Man ethos , an appealing blend of counter-cultural idealism and hedonistic creativity .',\n",
              "  'The movie is well done , but slow .',\n",
              "  'The fascination comes in the power of the Huston performance , which seems so larger than life and yet so fragile , and in the way the Ivan character accepts the news of his illness so quickly but still finds himself unable to react .',\n",
              "  'The last scenes of the film are anguished , bitter and truthful .',\n",
              "  'Mr. Koshashvili is a director to watch .',\n",
              "  'Predictable storyline and by-the-book scripting is all but washed away by sumptuous ocean visuals and the cinematic stylings of director John Stockwell .',\n",
              "  'Antwone Fisher certainly does the trick of making us care about its protagonist and celebrate his victories but , with few exceptions , it rarely stoops to cheap manipulation or corny conventions to do it .',\n",
              "  'One feels the dimming of a certain ambition , but in its place a sweetness , clarity and emotional openness that recalls the classics of early Italian neorealism .',\n",
              "  'It challenges , this nervy oddity , like modern art should .',\n",
              "  \"Whenever you think you 've figured out Late Marriage , it throws you for a loop .\",\n",
              "  \"The Pianist is Polanski 's best film .\",\n",
              "  \"This Orange has some juice , but it 's far from fresh-squeezed .\",\n",
              "  'A sensitive , modest comic tragedy that works as both character study and symbolic examination of the huge economic changes sweeping modern China .',\n",
              "  \"Cusack 's just brilliant in this .\",\n",
              "  'Knows how to make our imagination wonder .',\n",
              "  \"Jae-eun Jeong 's Take Care of My Cat brings a beguiling freshness to a coming-of-age story with such a buoyant , expressive flow of images that it emerges as another key contribution to the flowering of the South Korean cinema .\",\n",
              "  'The overall fabric is hypnotic , and Mr. Mattei fosters moments of spontaneous intimacy .',\n",
              "  'Evokes a palpable sense of disconnection , made all the more poignant by the incessant use of cell phones .',\n",
              "  'Malcolm McDowell is cool .',\n",
              "  'Paul Bettany is cool .',\n",
              "  'Cool .',\n",
              "  'A touching , sophisticated film that almost seems like a documentary in the way it captures an Italian immigrant family on the brink of major changes .',\n",
              "  \"An inventive , absorbing movie that 's as hard to classify as it is hard to resist .\",\n",
              "  'It made me want to get made-up and go see this movie with my sisters .',\n",
              "  \"I thought the relationships were wonderful , the comedy was funny , and the love ` real ' .\",\n",
              "  \"Bogdanovich puts history in perspective and , via Kirsten Dunst 's remarkable performance , he showcases Davies as a young woman of great charm , generosity and diplomacy .\",\n",
              "  'This breezy caper movie becomes a soulful , incisive meditation on the way we were , and the way we are .',\n",
              "  'A captivating new film .',\n",
              "  \"Those who are n't put off by the film 's austerity will find it more than capable of rewarding them .\",\n",
              "  \"It 's a clear-eyed portrait of an intensely lived time , filled with nervous energy , moral ambiguity and great uncertainties .\",\n",
              "  'Reveals how important our special talents can be when put in service of of others .',\n",
              "  'It also shows how deeply felt emotions can draw people together across the walls that might otherwise separate them .',\n",
              "  'With the same sort of good-natured fun found in films like Tremors , Eight Legged Freaks is prime escapist fare .',\n",
              "  'A sharp , amusing study of the cult of celebrity .',\n",
              "  'The sentimental cliches mar an otherwise excellent film .',\n",
              "  'A powerful performance from Mel Gibson and a brutal 90-minute battle sequence that does everything but issue you a dog-tag and an M-16 .',\n",
              "  \"A graceful , moving tribute to the courage of New York 's finest and a nicely understated expression of the grief shared by the nation at their sacrifice .\",\n",
              "  'A coming-of-age tale from New Zealand whose boozy , languid air is balanced by a rich visual clarity and deeply felt performances across the board .',\n",
              "  'Made to be Jaglomized is the Cannes Film Festival , the annual Riviera spree of flesh , buzz , blab and money .',\n",
              "  'The charming result is Festival in Cannes .',\n",
              "  \"If you 're looking for something new and hoping for something entertaining , you 're in luck .\",\n",
              "  \"A hugely rewarding experience that 's every bit as enlightening , insightful and entertaining as Grant 's two best films -- Four Weddings and a Funeral and Bridget Jones 's Diary .\",\n",
              "  \"A rip-roaring comedy action fest that 'll put hairs on your chest .\",\n",
              "  \"If there 's no art here , it 's still a good yarn -- which is nothing to sneeze at these days .\",\n",
              "  'Simultaneously heart-breaking and very funny , The Last Kiss is really all about performances .',\n",
              "  'There is a subversive element to this Disney cartoon , providing unexpected fizzability .',\n",
              "  'An unforgettable look at morality , family , and social expectation through the prism of that omnibus tradition called marriage .',\n",
              "  'An enjoyable , if occasionally flawed , experiment .',\n",
              "  \"Miyazaki is one of world cinema 's most wondrously gifted artists and storytellers .\",\n",
              "  'If Ayurveda can help us return to a sane regimen of eating , sleeping and stress-reducing contemplation , it is clearly a good thing .',\n",
              "  \"Meeting , even exceeding expectations , it 's the best sequel since The Empire Strikes Back ... a majestic achievement , an epic of astonishing grandeur and surprising emotional depth .\",\n",
              "  'Leigh is one of the rare directors who feels acting is the heart and soul of cinema .',\n",
              "  'He allows his cast members to make creative contributions to the story and dialogue .',\n",
              "  'This method almost never fails him , and it works superbly here .',\n",
              "  'Poetry in motion captured on film .',\n",
              "  \"While it can be a bit repetitive , overall it 's an entertaining and informative documentary .\",\n",
              "  'The entire movie establishes a wonderfully creepy mood .',\n",
              "  'I found The Ring moderately absorbing , largely for its elegantly colorful look and sound .',\n",
              "  'The filmmakers want nothing else than to show us a good time , and in their cheap , B movie way , they succeed .',\n",
              "  'Amari has dressed up this little parable in a fairly irresistible package full of privileged moments and memorable performances .',\n",
              "  \"But it will just as likely make you weep , and it will do so in a way that does n't make you feel like a sucker .\",\n",
              "  \"Both heartbreaking and heartwarming ... just a simple fable done in an artless sytle , but it 's tremendously moving .\",\n",
              "  'This masterfully calibrated psychological thriller thrives on its taut performances and creepy atmosphere even if the screenplay falls somewhat short .',\n",
              "  \"The film 's sense of imagery gives it a terrible strength , but it 's propelled by the acting .\",\n",
              "  \"Here is a divine monument to a single man 's struggle to regain his life , his dignity and his music .\",\n",
              "  'Strange it is , but delightfully so .',\n",
              "  'Elegant , mannered and teasing .',\n",
              "  'The ingenuity that Parker displays in freshening the play is almost in a class with that of Wilde himself .',\n",
              "  'The unexpected thing is that its dying , in this shower of black-and-white psychedelia , is quite beautiful .',\n",
              "  'A droll , bitchy frolic which pokes fun at the price of popularity and small-town pretension in the Lone Star State .',\n",
              "  \"With each of her three protagonists , Miller eloquently captures the moment when a woman 's life , out of a deep-seated , emotional need , is about to turn onto a different path .\",\n",
              "  'Ryan Gosling ... is at 22 a powerful young actor .',\n",
              "  \"As an introduction to the man 's theories and influence , Derrida is all but useless ; as a portrait of the artist as an endlessly inquisitive old man , however , it 's invaluable .\",\n",
              "  'The film is a verbal duel between two gifted performers .',\n",
              "  'Imperfect ?',\n",
              "  'Yes , but also intriguing and honorable , a worthwhile addition to a distinguished film legacy .',\n",
              "  'What a great way to spend 4 units of your day .',\n",
              "  \"The movie is hardly a masterpiece , but it does mark Ms. Bullock 's best work in some time .\",\n",
              "  'As simple and innocent a movie as you can imagine .',\n",
              "  'This is a movie you can trust .',\n",
              "  'Passionate , irrational , long-suffering but cruel as a tarantula , Helga figures prominently in this movie , and helps keep the proceedings as funny for grown-ups as for rugrats .',\n",
              "  \"Vividly conveys the passion , creativity , and fearlessness of one of Mexico 's most colorful and controversial artists -- a captivating drama that will speak to the nonconformist in us all .\",\n",
              "  'Hollywood Ending is not show-stoppingly hilarious , but scathingly witty nonetheless .',\n",
              "  'A compelling yarn , but not quite a ripping one .',\n",
              "  'On the Granger Movie Gauge of 1 to 10 , The Powerpuff Girls is a fast , frenetic , funny , even punny 6 -- aimed specifically at a grade-school audience .',\n",
              "  'The film has several strong performances .',\n",
              "  \"I 've never bought from telemarketers , but I bought this movie .\",\n",
              "  'Perfectly pitched between comedy and tragedy , hope and despair , About Schmidt instead comes far closer than many movies to expressing the way many of us live -- someplace between consuming self-absorption and insistently demanding otherness .',\n",
              "  \"A sexy , peculiar and always entertaining costume drama set in Renaissance Spain , and the fact that it 's based on true events somehow makes it all the more compelling .\",\n",
              "  \"An entertaining documentary that freshly considers arguments the Bard 's immortal plays were written by somebody else .\",\n",
              "  \"A highly spirited , imaginative kid 's movie that broaches neo-Augustinian theology : Is God stuck in Heaven because He 's afraid of His best-known creation ?\",\n",
              "  'Call it magic realism or surrealism , but Miss Wonton floats beyond reality with a certain degree of wit and dignity .',\n",
              "  \"Raimi and his team could n't have done any better in bringing the story of Spider-Man to the big screen .\",\n",
              "  'The director explores all three sides of his story with a sensitivity and an inquisitiveness reminiscent of Truffaut .',\n",
              "  'Well-acted , well-directed and , for all its moodiness , not too pretentious .',\n",
              "  \"It 's a satisfying summer blockbuster and worth a look .\",\n",
              "  'Boomers and their kids will have a Barrie good time .',\n",
              "  'Real Women Have Curves wears its empowerment on its sleeve but even its worst harangues are easy to swallow thanks to remarkable performances by Ferrera and Ontiveros .',\n",
              "  \"Ultimately , `` MIB II '' succeeds due to its rapid-fire delivery and enough inspired levity that it ca n't be dismissed as mindless .\",\n",
              "  'Stage director Sam Mendes showcases Tom Hanks as a depression era hit-man in this dark tale of revenge .',\n",
              "  \"Sitting in the third row of the IMAX cinema at Sydney 's Darling Harbour , but I sometimes felt as though I was in the tiny two seater plane that carried the giant camera around Australia , sweeping and gliding , banking and hovering over some of the most not\",\n",
              "  \"What 's so fun about this silly , outrageous , ingenious thriller is the director 's talent .\",\n",
              "  'Watching a Brian DePalma movie is like watching an Alfred Hitchcock movie after drinking twelve beers .',\n",
              "  \"Strip it of all its excess debris , and you 'd have a 90-minute , four-star movie .\",\n",
              "  \"As it is , it 's too long and unfocused .\",\n",
              "  'An immensely entertaining look at some of the unsung heroes of 20th century pop music .',\n",
              "  \"This familiar rise-and-fall tale is long on glamour and short on larger moralistic consequences , though it 's told with sharp ears and eyes for the tenor of the times .\",\n",
              "  'This beautifully animated epic is never dull .',\n",
              "  \"Brian Tufano 's handsome widescreen photography and Paul Grabowsky 's excellent music turn this fairly parochial melodrama into something really rather special .\",\n",
              "  'It makes compelling , provocative and prescient viewing .',\n",
              "  \"A thoroughly entertaining comedy that uses Grant 's own twist of acidity to prevent itself from succumbing to its own bathos .\",\n",
              "  'Using a stock plot , About a Boy injects just enough freshness into the proceedings to provide an enjoyable 100 minutes in a movie theater .',\n",
              "  \"But it 's defiantly and delightfully against the grain .\",\n",
              "  'The hard-to-predict and absolutely essential chemistry between the down-to-earth Bullock and the nonchalant Grant proves to be sensational , and everything meshes in this elegant entertainment .',\n",
              "  'A positively thrilling combination of ethnography and all the intrigue , betrayal , deceit and murder of a Shakespearean tragedy or a juicy soap opera .',\n",
              "  'Mr. Clooney , Mr. Kaufman and all their collaborators are entitled to take a deep bow for fashioning an engrossing entertainment out of an almost sure-fire prescription for a critical and commercial disaster .',\n",
              "  \"Definitely funny stuff , but it 's more of the ` laughing at ' variety than the ` laughing with . '\",\n",
              "  'Easily the most thoughtful fictional examination of the root causes of anti-Semitism ever seen on screen .',\n",
              "  'A real winner -- smart , funny , subtle , and resonant .',\n",
              "  'Family portrait of need , neurosis and nervy negativity is a rare treat that shows the promise of digital filmmaking .',\n",
              "  'Uplifting as only a document of the worst possibilities of mankind can be , and among the best films of the year .',\n",
              "  'The film has a terrific look and Salma Hayek has a feel for the character at all stages of her life .',\n",
              "  'A decided lack of spontaneity in its execution and a dearth of real poignancy in its epiphanies .',\n",
              "  'The performances are remarkable .',\n",
              "  \"It 's Burns ' visuals , characters and his punchy dialogue , not his plot , that carry waydowntown .\",\n",
              "  \"As literary desecrations go , this makes for perfectly acceptable , occasionally very enjoyable children 's entertainment .\",\n",
              "  \"You 'll forget about it by Monday , though , and if they 're old enough to have developed some taste , so will your kids .\",\n",
              "  \"While I ca n't say it 's on par with the first one , Stuart Little 2 is a light , fun cheese puff of a movie .\",\n",
              "  'Strange , funny , twisted , brilliant and macabre .',\n",
              "  'A genuinely moving and wisely unsentimental drama .',\n",
              "  'The Mothman Prophecies is best when illustrating the demons bedevilling the modern masculine journey .',\n",
              "  'Plays out with a dogged and eventually winning squareness that would make it the darling of many a kids-and-family-oriented cable channel .',\n",
              "  'An entertaining British hybrid of comedy , caper thrills and quirky romance .',\n",
              "  \"The only pain you 'll feel as the credits roll is your stomach grumbling for some tasty grub .\",\n",
              "  \"Hardly an objective documentary , but it 's great cinematic polemic ... love Moore or loathe him , you 've got to admire ... the intensity with which he 's willing to express his convictions .\",\n",
              "  'The mark of a respectable summer blockbuster is one of two things : unadulterated thrills or genuine laughs .',\n",
              "  'The film is visually dazzling , the depicted events dramatic , funny and poignant .',\n",
              "  'A directorial tour de force by Bernard Rose , ivans xtc .',\n",
              "  \"is one of this year 's very best pictures .\",\n",
              "  'Weaver and LaPaglia are both excellent , in the kind of low-key way that allows us to forget that they are actually movie folk .',\n",
              "  'Even the digressions are funny .',\n",
              "  'Mr. Spielberg and his company just want you to enjoy yourselves without feeling conned .',\n",
              "  'And they succeed merrily at their noble endeavor .',\n",
              "  'A perfectly pleasant if slightly pokey comedy .',\n",
              "  \"Coppola 's directorial debut is an incredibly layered and stylistic film that , despite a fairly slow paced , almost humdrum approach to character development , still manages at least a decent attempt at meaningful cinema .\",\n",
              "  'At the end , when the now computerized Yoda finally reveals his martial artistry , the film ascends to a kinetic life so teeming that even cranky adults may rediscover the quivering kid inside .',\n",
              "  'Wang Xiaoshuai directs this intricately structured and well-realized drama that presents a fascinating glimpse of urban life and the class warfare that embroils two young men .',\n",
              "  \"It 's hard to imagine anybody ever being `` in the mood '' to view a movie as harrowing and painful as The Grey Zone , but it 's equally hard to imagine anybody being able to tear their eyes away from the screen once it 's started .\",\n",
              "  'Bogdanovich taps deep into the Hearst mystique , entertainingly reenacting a historic scandal .',\n",
              "  'A moving tale of love and destruction in unexpected places , unexamined lives .',\n",
              "  'Clooney directs this film always keeping the balance between the fantastic and the believable ...',\n",
              "  \"Even if you do n't understand what on earth is going on , this is a movie that will stimulate hours of post viewing discussion , if only to be reminded of who did what to whom and why .\",\n",
              "  'A fantastically vital movie that manages to invest real humor , sensuality , and sympathy into a story about two adolescent boys .',\n",
              "  'Lawrence plumbs personal tragedy and also the human comedy .',\n",
              "  'Though a capable thriller , somewhere along the way K-19 jettisoned some crucial drama .',\n",
              "  'Just about the surest bet for an all-around good time at the movies this summer .',\n",
              "  'This is a movie that is what it is : a pleasant distraction , a Friday night diversion , an excuse to eat popcorn .',\n",
              "  'There is a certain sense of experimentation and improvisation to this film that may not always work , but it is nevertheless compelling .',\n",
              "  'The Four Feathers has rewards , from the exoticism of its seas of sand to the fierce grandeur of its sweeping battle scenes .',\n",
              "  'A delicious , quirky movie with a terrific screenplay and fanciful direction by Michael Gondry .',\n",
              "  'This story still seems timely and important .',\n",
              "  \"And there 's an element of heartbreak to watching it now , with older and wiser eyes , because we know what will happen after Greene 's story ends .\",\n",
              "  \"The bodily function jokes are about what you 'd expect , but there are rich veins of funny stuff in this movie .\",\n",
              "  'The performances are amiable and committed , and the comedy more often than not hits the bullseye .',\n",
              "  \"This time , the hype is quieter , and while the movie is slightly less successful than the first , it 's still a rollicking good time for the most part .\",\n",
              "  \"There 's plenty to enjoy -- in no small part thanks to Lau .\",\n",
              "  \"With a romantic comedy plotline straight from the ages , this Cinderella story does n't have a single surprise up its sleeve .\",\n",
              "  'But it does somehow manage to get you under its spell .',\n",
              "  \"Though few will argue that it ranks with the best of Herzog 's works , Invincible shows he 's back in form , with an astoundingly rich film .\",\n",
              "  \"`` Catch Me '' feels capable of charming the masses with star power , a pop-induced score and sentimental moments that have become a Spielberg trademark .\",\n",
              "  'By no means a great movie , but it is a refreshingly forthright one .',\n",
              "  \"The casting of Raymond J. Barry as the ` assassin ' greatly enhances the quality of Neil Burger 's impressive fake documentary .\",\n",
              "  \"Despite Besson 's high-profile name being Wasabi 's big selling point , there is no doubt that Krawczyk deserves a huge amount of the credit for the film 's thoroughly winning tone .\",\n",
              "  \"A thoughtful movie , a movie that is concerned with souls and risk and schemes and the consequences of one 's actions .\",\n",
              "  'As satisfyingly odd and intriguing a tale as it was a century and a half ago ... has a delightfully dour , deadpan tone and stylistic consistency .',\n",
              "  \"Methodical , measured , and gently tedious in its comedy , Secret Ballot is a purposefully reductive movie -- which may be why it 's so successful at lodging itself in the brain .\",\n",
              "  'For most of the distance the picture provides a satisfyingly unsettling ride into the dark places of our national psyche .',\n",
              "  'By the standards of knucklehead swill , The Hot Chick is pretty damned funny .',\n",
              "  'One of the most gloriously unsubtle and adrenalized extreme shockers since The Evil Dead .',\n",
              "  'An absorbing and unsettling psychological drama .',\n",
              "  \"This movie may not have the highest production values you 've ever seen , but it 's the work of an artist , one whose view of America , history and the awkwardness of human life is generous and deep .\",\n",
              "  \"Though it 's not very well shot or composed or edited , the score is too insistent and the dialogue is frequently overwrought and crudely literal , the film shatters you in waves .\",\n",
              "  'The entire cast is extraordinarily good .',\n",
              "  'Yakusho , as always , is wonderful as the long-faced sad sack ... and his chemistry with Shimizu is very believable .',\n",
              "  'Young Hanks and Fisk , who vaguely resemble their celebrity parents , bring fresh good looks and an ease in front of the camera to the work .',\n",
              "  'A captivatingly quirky hybrid of character portrait , romantic comedy and beat-the-clock thriller .',\n",
              "  'The film sparkles with the the wisdom and humor of its subjects .',\n",
              "  \"Like The English Patient and The Unbearable Lightness of Being , The Hours is one of those reputedly `` unfilmable '' novels that has bucked the odds to emerge as an exquisite motion picture in its own right .\",\n",
              "  'Just about the best straight-up , old-school horror film of the last 15 years .',\n",
              "  'A chilling tale of one of the great crimes of 20th Century France : the murder of two rich women by their servants in 1933 .',\n",
              "  'An oddity , to be sure , but one that you might wind up remembering with a degree of affection rather than revulsion .',\n",
              "  'While the film is not entirely successful , it still manages to string together enough charming moments to work .',\n",
              "  'A winning piece of work filled with love for the movies of the 1960s .',\n",
              "  'E.T. works because its flabbergasting principals , 14-year-old Robert MacNaughton , 6-year-old Drew Barrymore and 10-year-old Henry Thomas , convince us of the existence of the wise , wizened visitor from a faraway planet .',\n",
              "  'A heartening tale of small victories and enduring hope .',\n",
              "  'Jackson and co have brought back the value and respect for the term epic cinema .',\n",
              "  \"It may be a somewhat backhanded compliment to say that the film makes the viewer feel like the movie 's various victimized audience members after a while , but it also happens to be the movie 's most admirable quality\",\n",
              "  'Charlotte Sometimes is a brilliant movie .',\n",
              "  'It is about irrational , unexplainable life and it seems so real because it does not attempt to filter out the complexity .',\n",
              "  \"A delightful stimulus for the optic nerves , so much that it 's forgivable that the plot feels like every other tale of a totalitarian tomorrow .\",\n",
              "  \"But darned if it does n't also keep us riveted to our seats .\",\n",
              "  \"A complex psychological drama about a father who returns to his son 's home after decades away .\",\n",
              "  \"Writer and director Otar Iosseliani 's pleasant tale about a factory worker who escapes for a holiday in Venice reveals how we all need a playful respite from the grind to refresh our souls .\",\n",
              "  'Sweet and memorable film .',\n",
              "  'A smart , arch and rather cold-blooded comedy .',\n",
              "  'Keenly observed and refreshingly natural , Swimming gets the details right , from its promenade of barely clad bodies in Myrtle Beach , S.C. , to the adrenaline jolt of a sudden lunch rush at the diner .',\n",
              "  '... begins on a high note and sustains it beautifully .',\n",
              "  'Davis ... gets vivid performances from her cast and pulls off some deft Ally McBeal-style fantasy sequences .',\n",
              "  \"` it 's better to go in knowing full well what 's going to happen , but willing to let the earnestness of its execution and skill of its cast take you down a familiar road with a few twists .\",\n",
              "  'Funny , somber , absurd , and , finally , achingly sad , Bartleby is a fine , understated piece of filmmaking .',\n",
              "  \"`` Red Dragon '' is entertaining .\",\n",
              "  'An obvious copy of one of the best films ever made , how could it not be ?',\n",
              "  'But it is entertaining on an inferior level .',\n",
              "  'It is a popcorn film , not a must-own , or even a must-see .',\n",
              "  \"On that score , the film certainly does n't disappoint .\",\n",
              "  \"As the movie traces Mr. Brown 's athletic exploits , it is impossible not to be awed by the power and grace of one of the greatest natural sportsmen of modern times .\",\n",
              "  'Thanks to confident filmmaking and a pair of fascinating performances , the way to that destination is a really special walk in the woods .',\n",
              "  \"Beautifully shot , delicately scored and powered by a set of heartfelt performances , it 's a lyrical endeavour .\",\n",
              "  'A macabre and very stylized Swedish fillm about a modern city where all the religious and civic virtues that hold society in place are in tatters .',\n",
              "  \"A stylistic romp that 's always fun to watch .\",\n",
              "  \"Informative , intriguing , observant , often touching ... gives a human face to what 's often discussed in purely abstract terms .\",\n",
              "  '... once the true impact of the day unfolds , the power of this movie is undeniable .',\n",
              "  'An honest , sensitive story from a Vietnamese point of view .',\n",
              "  \"A buoyant romantic comedy about friendship , love , and the truth that we 're all in this together .\",\n",
              "  \"The film 's intimate camera work and searing performances pull us deep into the girls ' confusion and pain as they struggle tragically to comprehend the chasm of knowledge that 's opened between them .\",\n",
              "  \"It 's the perfect star vehicle for Grant , allowing him to finally move away from his usual bumbling , tongue-tied screen persona .\",\n",
              "  'Not only a reminder of how they used to make movies , but also how they sometimes still can be made .',\n",
              "  'A three-hour cinema master class .',\n",
              "  'Eyre is on his way to becoming the American Indian Spike Lee .',\n",
              "  'A witty , whimsical feature debut .',\n",
              "  'Warm in its loving yet unforgivingly inconsistent depiction of everyday people , relaxed in its perfect quiet pace and proud in its message .',\n",
              "  'I loved this film .',\n",
              "  \"Clint Eastwood 's Blood Work is a lot like a well-made PB & J sandwich : familiar , fairly uneventful and boasting no real surprises -- but still quite tasty and inviting all the same .\",\n",
              "  \"A movie that will surely be profane , politically charged music to the ears of Cho 's fans .\",\n",
              "  'Much of this slick and sprightly CGI feature is sufficiently funny to amuse even the most resolutely unreligious parents who escort their little ones to megaplex screenings .',\n",
              "  'Rarely , a movie is more than a movie .',\n",
              "  \"Jacquot 's strategy allows his cast the benefit of being able to give full performances ... while demonstrating vividly that the beauty and power of the opera reside primarily in the music itself .\",\n",
              "  'By not averting his eyes , Solondz forces us to consider the unthinkable , the unacceptable , the unmentionable .',\n",
              "  \"One Hour Photo may seem disappointing in its generalities , but it 's the little nuances that perhaps had to escape from director Mark Romanek 's self-conscious scrutiny to happen , that finally get under your skin .\",\n",
              "  \"While general audiences might not come away with a greater knowledge of the facts of Cuban music , they 'll be treated to an impressive and highly entertaining celebration of its sounds .\",\n",
              "  'A fascinating documentary that provides a rounded and revealing overview of this ancient holistic healing system',\n",
              "  \"Birthday Girl lucks out with Chaplin and Kidman , who are capable of anteing up some movie star charisma when they need it to sell us on this twisted love story , but who can also negotiate the movie 's darker turns .\",\n",
              "  'An interesting look behind the scenes of Chicago-based rock group Wilco ...',\n",
              "  'Sharp edges and a deep vein of sadness run through its otherwise comic narrative .',\n",
              "  \"There 's lots of cool stuff packed into ESPN 's Ultimate X.\",\n",
              "  'Rock solid family fun out of the gates , extremely imaginative through out , but wanes in the middle',\n",
              "  \"The Ya-Ya 's have many secrets and one is - the books are better .\",\n",
              "  'Translating complex characters from novels to the big screen is an impossible task but they are true to the essence of what it is to be Ya-Ya .',\n",
              "  'The touch is generally light enough and the performances , for the most part , credible .',\n",
              "  'I liked About Schmidt a lot , but I have a feeling that I would have liked it much more if Harry & Tonto never existed .',\n",
              "  'That Storytelling has value can not be denied .',\n",
              "  \"Not even Solondz 's thirst for controversy , sketchy characters and immature provocations can fully succeed at cheapening it .\",\n",
              "  'Once the downward spiral comes to pass , Auto Focus bears out as your typical junkie opera ...',\n",
              "  'A knowing sense of humor and a lot of warmth ignite Son of the Bride .',\n",
              "  'A rich tale of our times , very well told with an appropriate minimum of means .',\n",
              "  'The characters are complex and quirky , but entirely believable as the remarkable ensemble cast brings them to life .',\n",
              "  'In all fairness , I must report that the children of varying ages in my audience never coughed , fidgeted or romped up and down the aisles for bathroom breaks .',\n",
              "  'As gory as the scenes of torture and self-mutilation may be , they are pitted against shimmering cinematography that lends the setting the ethereal beauty of an Asian landscape painting .',\n",
              "  'Efficient , suitably anonymous chiller .',\n",
              "  'The hot topics of the plot are relegated to the background -- a welcome step forward from the Sally Jesse Raphael atmosphere of films like Philadelphia and American Beauty .',\n",
              "  \"It 's usually a bad sign when directors abandon their scripts and go where the moment takes them , but Olympia , Wash. , based filmmakers Anne de Marcken and Marilyn Freeman did just that and it 's what makes their project so interesting .\",\n",
              "  'A memorable experience that , like many of his works , presents weighty issues colorfully wrapped up in his own idiosyncratic strain of kitschy goodwill .',\n",
              "  'Executed with such gentle but insistent sincerity , with such good humor and appreciation of the daily grind that only the most hardhearted Scrooge could fail to respond .',\n",
              "  \"The gentle comic treatment of adolescent sturm und drang should please fans of Chris Fuhrman 's posthumously published cult novel .\",\n",
              "  'Director Claude Chabrol has become the master of innuendo .',\n",
              "  'A deftly entertaining film , smartly played and smartly directed .',\n",
              "  'A documentary to make the stones weep -- as shameful as it is scary .',\n",
              "  'I hope the movie is widely seen and debated with appropriate ferocity and thoughtfulness .',\n",
              "  \"Asks what truth can be discerned from non-firsthand experience , and specifically questions cinema 's capability for recording truth .\",\n",
              "  \"The journey to the secret 's eventual discovery is a separate adventure , and thrill enough .\",\n",
              "  'A quiet , disquieting triumph .',\n",
              "  'Darkly funny and frequently insightful .',\n",
              "  \"... the tale of her passionate , tumultuous affair with Musset unfolds as Sand 's masculine persona , with its love of life and beauty , takes form .\",\n",
              "  \"There 's so much to look at in Metropolis you hate to tear your eyes away from the images long enough to read the subtitles .\",\n",
              "  \"The search for redemption makes for a touching love story , mainly because Blanchett and Ribisi compellingly tap into a spiritual aspect of their characters ' suffering .\",\n",
              "  'A film of ideas and wry comic mayhem .',\n",
              "  \"A worthwhile documentary , whether you 're into rap or not , even if it may still leave you wanting more answers as the credits roll .\",\n",
              "  \"Fessenden 's narrative is just as much about the ownership and redefinition of myth as it is about a domestic unit finding their way to joy .\",\n",
              "  'That the film opens with maggots crawling on a dead dog is not an out of place metaphor .',\n",
              "  'Stanley Kwan has directed not only one of the best gay love stories ever made , but one of the best love stories of any stripe .',\n",
              "  'The concert footage is stirring , the recording sessions are intriguing , and -- on the way to striking a blow for artistic integrity -- this quality band may pick up new admirers .',\n",
              "  \"You wo n't have any trouble getting kids to eat up these Veggies .\",\n",
              "  \"Enjoyably dumb , sweet , and intermittently hilarious -- if you 've a taste for the quirky , steal a glimpse .\",\n",
              "  \"A movie that sends you out of the theater feeling like you 've actually spent time living in another community .\",\n",
              "  'Light-years ahead of paint-by-number American blockbusters like Pearl Harbor , at least artistically .',\n",
              "  'A fascinating documentary about the long and eventful spiritual journey of the guru who helped launch the New Age .',\n",
              "  \"Isabelle Huppert excels as the enigmatic Mika and Anna Mouglalis is a stunning new young talent in one of Chabrol 's most intense psychological mysteries .\",\n",
              "  'Wedding feels a bit anachronistic .',\n",
              "  \"It 's still a comic book , but Maguire makes it a comic book with soul .\",\n",
              "  'Brings to a spectacular completion one of the most complex , generous and subversive artworks of the last decade .',\n",
              "  'An amusing and unexpectedly insightful examination of sexual jealousy , resentment and the fine line between passion and pretence .',\n",
              "  'A fascinating , bombshell documentary that should shame Americans , regardless of whether or not ultimate blame finally lies with Kissinger .',\n",
              "  'Should be required viewing for civics classes and would-be public servants alike .',\n",
              "  \"Adaptation 's success in engaging the audience in the travails of creating a screenplay is extraordinary .\",\n",
              "  'A polished and vastly entertaining caper film that puts the sting back into the con .',\n",
              "  \"It 's no surprise that as a director Washington demands and receives excellent performances , from himself and from newcomer Derek Luke .\",\n",
              "  \"... while each moment of this broken character study is rich in emotional texture , the journey does n't really go anywhere .\",\n",
              "  'The film gets close to the chimps the same way Goodall did , with a serious minded patience , respect and affection .',\n",
              "  \"This may be Burns 's strongest film since The Brothers McMullen .\",\n",
              "  \"What makes this film special is Serry 's ability to take what is essentially a contained family conflict and put it into a much larger historical context .\",\n",
              "  \"It 's Quaid who anchors the film with his effortless performance and that trademark grin of his -- so perfect for a ballplayer .\",\n",
              "  'It is OK for a movie to be something of a sitcom apparatus , if the lines work , the humor has point and the actors are humanly engaged .',\n",
              "  'A sly female empowerment movie , although not in a way anyone would expect .',\n",
              "  \"It 's a wonderful , sobering , heart-felt drama .\",\n",
              "  \"Runs on the pure adrenalin of Pacino 's performance .\",\n",
              "  \"Phillip Noyce and all of his actors -- as well as his cinematographer , Christopher Doyle -- understand the delicate forcefulness of Greene 's prose , and it 's there on the screen in their version of The Quiet American .\",\n",
              "  'The film just might turn on many people to opera , in general , an art form at once visceral and spiritual , wonderfully vulgar and sublimely lofty -- and as emotionally grand as life .',\n",
              "  \"As a vehicle to savour Binoche 's skill , the film is well worthwhile .\",\n",
              "  'The huskies are beautiful , the border collie is funny and the overall feeling is genial and decent .',\n",
              "  'Mastering its formidable arithmetic of cameras and souls , Group articulates a flood of emotion .',\n",
              "  'A pretty decent kid-pleasing , tolerable-to-adults lark of a movie .',\n",
              "  'Even during the climactic hourlong cricket match , boredom never takes hold .',\n",
              "  \"Combine the paranoid claustrophobia of a submarine movie with the unsettling spookiness of the supernatural -- why did n't Hollywood think of this sooner ?\",\n",
              "  \"Like Kubrick , Soderbergh is n't afraid to try any genre and to do it his own way .\",\n",
              "  'Nothing can detract from the affection of that moral favorite : friends will be friends through thick and thin .',\n",
              "  'If the film has a problem , its shortness disappoints : You want the story to go on and on .',\n",
              "  'Unlike most anime , whose most ardent fans outside Japan seem to be introverted young men with fantasy fetishes , Metropolis never seems hopelessly juvenile .',\n",
              "  'The plot twists give I Am Trying to Break Your Heart an attraction it desperately needed .',\n",
              "  'The most brilliant and brutal UK crime film since Jack Carter went back to Newcastle , the first half of Gangster No. 1 drips with style and , at times , blood .',\n",
              "  'Like its New England characters , most of whom wander about in thick clouds of denial , the movie eventually gets around to its real emotional business , striking deep chords of sadness .',\n",
              "  \"The Bai brothers have taken an small slice of history and opened it up for all of us to understand , and they 've told a nice little story in the process .\",\n",
              "  \"Flamboyant in some movies and artfully restrained in others , 65-year-old Jack Nicholson could be looking at his 12th Oscar nomination by proving that he 's now , more than ever , choosing his roles with the precision of the insurance actuary .\",\n",
              "  \"The most amazing super-sized dosage of goofball stunts any `` Jackass '' fan could want .\",\n",
              "  'Real Women may have many agendas , but it also will win you over , in a big way .',\n",
              "  'Young Everlyn Sampi , as the courageous Molly Craig , simply radiates star-power potential in this remarkable and memorable film .',\n",
              "  'Surprisingly powerful and universal .',\n",
              "  'Morvern Callar confirms Lynne Ramsay as an important , original talent in international cinema .',\n",
              "  'Well-done supernatural thriller with keen insights into parapsychological phenomena and the soulful nuances of the grieving process .',\n",
              "  \"A plethora of engaging diatribes on the meaning of ` home , ' delivered in grand passion by the members of the various households .\",\n",
              "  \"It 's technically sumptuous but also almost wildly alive .\",\n",
              "  \"This film puts Wang at the forefront of China 's Sixth Generation of film makers .\",\n",
              "  \"it 's refreshing to see a movie that embraces its old-fashioned themes and in the process comes out looking like something wholly original .\",\n",
              "  'Wiseman is patient and uncompromising , letting his camera observe and record the lives of women torn apart by a legacy of abuse .',\n",
              "  \"There 's none of the happily-ever - after spangle of Monsoon Wedding in Late Marriage -- and that 's part of what makes Dover Kosashvili 's outstanding feature debut so potent .\",\n",
              "  'An ingenious and often harrowing look at damaged people and how families can offer either despair or consolation .',\n",
              "  'Arguably the best script that Besson has written in years .',\n",
              "  \"It 's no lie -- Big Fat Liar is a real charmer .\",\n",
              "  'Invigorating , surreal , and resonant with a rainbow of emotion .',\n",
              "  'Director Alfonso Cuaron gets vivid , convincing performances from a fine cast , and generally keeps things going at a rapid pace , occasionally using an omniscient voice-over narrator in the manner of French New Wave films .',\n",
              "  'Pray has really done his subject justice .',\n",
              "  'An unexpectedly sweet story of sisterhood .',\n",
              "  'Maintains your sympathy for this otherwise challenging soul by letting you share her one-room world for a while .',\n",
              "  'A subtle , humorous , illuminating study of politics , power and social mobility .',\n",
              "  '- West Coast rap wars , this modern mob music drama never fails to fascinate .',\n",
              "  \"Nair 's attention to detail creates an impeccable sense of place , while Thurman and Lewis give what can easily be considered career-best performances .\",\n",
              "  \"Berry 's saucy , full-bodied performance gives this aging series a much needed kick , making `` Die Another Day '' one of the most entertaining Bonds in years\",\n",
              "  \"But , like Silence , it 's a movie that gets under your skin .\",\n",
              "  'Caviezel embodies the transformation of his character completely .',\n",
              "  'A creepy , intermittently powerful study of a self-destructive man ... about as unsettling to watch as an exploratory medical procedure or an autopsy .',\n",
              "  'Pacino and Williams seem to keep upping the ante on each other , just as their characters do in the film .',\n",
              "  'What results is the best performance from either in years .',\n",
              "  'The cast is top-notch and I predict there will be plenty of female audience members drooling over Michael Idemoto as Michael .',\n",
              "  \"Miyazaki has created such a vibrant , colorful world , it 's almost impossible not to be swept away by the sheer beauty of his images .\",\n",
              "  \"Muccino seems to be exploring the idea of why human beings long for what they do n't have , and how this gets us in trouble .\",\n",
              "  'But even while his characters are acting horribly , he is always sympathetic .',\n",
              "  \"Whether or not you buy Mr. Broomfield 's findings , the film acquires an undeniable entertainment value as the slight , pale Mr. Broomfield continues to force himself on people and into situations that would make lesser men run for cover .\",\n",
              "  '... an eerily suspenseful , deeply absorbing piece that works as a treatise on spirituality as well as a solid sci-fi thriller .',\n",
              "  \"I 've never seen or heard anything quite like this film , and I recommend it for its originality alone .\",\n",
              "  'Nicole Kidman makes it a party worth attending .',\n",
              "  'The direction has a fluid , no-nonsense authority , and the performances by Harris , Phifer and Cam ` ron seal the deal .',\n",
              "  'The Komediant is a tale worth catching .',\n",
              "  'The writing is clever and the cast is appealing .',\n",
              "  'Spielberg has managed to marry science fiction with film noir and action flicks with philosophical inquiry .',\n",
              "  \"It 's the type of film about growing up that we do n't see often enough these days : realistic , urgent , and not sugarcoated in the least .\",\n",
              "  'A taut , sobering film .',\n",
              "  'Exudes the fizz of a Busby Berkeley musical and the visceral excitement of a sports extravaganza .',\n",
              "  \"It 's full of cheesy dialogue , but great trashy fun that finally returns De Palma to his pulpy thrillers of the early '80s .\",\n",
              "  'A quietly moving look back at what it was to be Iranian-American in 1979 .',\n",
              "  'Like a veteran head cutter , Barbershop is tuned in to its community .',\n",
              "  \"I 'm sure mainstream audiences will be baffled , but , for those with at least a minimal appreciation of Woolf and Clarissa Dalloway , The Hours represents two of those well spent .\",\n",
              "  'You live the mood rather than savour the story .',\n",
              "  'Angela Gheorghiu as famous prima donna Floria Tosca , Roberto Alagna as her lover Mario Cavaradossi , and Ruggero as the villainous , lecherous police chief Scarpia , all sing beautifully and act adequately .',\n",
              "  \"While there are times when the film 's reach exceeds its grasp , the production works more often than it does n't .\",\n",
              "  \"While Scorsese 's bold images and generally smart casting ensure that `` Gangs '' is never lethargic , the movie is hindered by a central plot that 's peppered with false starts and populated by characters who are nearly impossible to care about .\",\n",
              "  'Watching this gentle , mesmerizing portrait of a man coming to terms with time , you barely realize your mind is being blown .',\n",
              "  'The beautifully choreographed kitchen ballet is simple but absorbing .',\n",
              "  \"There 's ... an underlying Old World sexism to Monday Morning that undercuts its charm .\",\n",
              "  \"`` The best Disney movie since the Lion King ''\",\n",
              "  'Transcends its agenda to deliver awe-inspiring , at times sublime , visuals and offer a fascinating glimpse into the subculture of extreme athletes whose derring-do puts the X into the games .',\n",
              "  'Think of it as Gidget , only with muscles and a lot more smarts , but just as endearing and easy to watch .',\n",
              "  'There is no solace here , no entertainment value , merely a fierce lesson in where filmmaking can take us .',\n",
              "  'Giggling at the absurdities and inconsistencies is part of the fun .',\n",
              "  'But the talented cast alone will keep you watching , as will the fight scenes .',\n",
              "  'Arteta paints a picture of lives lived in a state of quiet desperation .',\n",
              "  \"Drug abuse , infidelity and death are n't usually comedy fare , but Turpin 's film allows us to chuckle through the angst .\",\n",
              "  'Efteriades gives the neighborhood -- scenery , vibe and all -- the cinematic equivalent of a big , tender hug .',\n",
              "  'Amazing !',\n",
              "  'A college story that works even without vulgarity , sex scenes , and cussing !',\n",
              "  \"The amazing film work is so convincing that by movies ' end you 'll swear you are wet in some places and feel sand creeping in others .\",\n",
              "  \"A raunchy and frequently hilarious follow-up to the gifted Korean American stand-up 's I 'm the One That I Want .\",\n",
              "  \"If you ever wanted to be an astronaut , this is the ultimate movie experience - it 's informative and breathtakingly spectacular .\",\n",
              "  \"While Parker and co-writer Catherine di Napoli are faithful to Melville 's plotline , they and a fully engaged supporting cast ... have made the old boy 's characters more quick-witted than any English Lit major would have thought possible .\",\n",
              "  'A smart , sassy and exceptionally charming romantic comedy .',\n",
              "  \"There are flaws , but also stretches of impact and moments of awe ; we 're wrapped up in the characters , how they make their choices , and why .\",\n",
              "  'A gift to anyone who loves both dance and cinema',\n",
              "  'A reminder that beyond all the hype and recent digital glitz , Spielberg knows how to tell us about people .',\n",
              "  'One of the finest , most humane and important Holocaust movies ever made .',\n",
              "  'An engrossing and infectiously enthusiastic documentary .',\n",
              "  'A beautiful , timeless and universal tale of heated passions -- jealousy , betrayal , forgiveness and murder .',\n",
              "  'A culture-clash comedy that , in addition to being very funny , captures some of the discomfort and embarrassment of being a bumbling American in Europe .',\n",
              "  'Shattering , devastating documentary on two maladjusted teens in a downward narcotized spiral .',\n",
              "  'Extraordinary debut from Josh Koury .',\n",
              "  'The most compelling performance of the year adds substantial depth to this shocking testament to anti-Semitism and neo-fascism .',\n",
              "  \"For those who are intrigued by politics of the '70s , the film is every bit as fascinating as it is flawed .\",\n",
              "  \"Douglas McGrath 's Nicholas Nickleby does Dickens as it should be done cinematically .\",\n",
              "  \"It 's a lovely , eerie film that casts an odd , rapt spell .\",\n",
              "  'The quirky and recessive charms of co-stars Martin Donovan and Mary-Louise Parker help overcome the problematic script .',\n",
              "  \"It 's good to see Michael Caine whipping out the dirty words and punching people in the stomach again .\",\n",
              "  \"It 's fun , wispy , wise and surprisingly inoffensive for a film about a teen in love with his stepmom .\",\n",
              "  'Able to provide insight into a fascinating part of theater history .',\n",
              "  'An unflinching , complex portrait of a modern Israel that is rarely seen on-screen .',\n",
              "  \"It 's a familiar story , but one that is presented with great sympathy and intelligence .\",\n",
              "  'Gently humorous and touching .',\n",
              "  \"It wo n't hold up over the long haul , but in the moment , Finch 's tale provides the forgettable pleasures of a Saturday matinee .\",\n",
              "  \"Kinnear 's performance is a career-defining revelation .\",\n",
              "  'The film is predictable in the reassuring manner of a beautifully sung holiday carol .',\n",
              "  \"... hits every cliche we 've come to expect , including the assumption that `` crazy '' people are innocent , childlike and inherently funny .\",\n",
              "  'A certain sexiness underlines even the dullest tangents .',\n",
              "  'You may be captivated , as I was , by its moods , and by its subtly transformed star , and still wonder why Paul Thomas Anderson ever had the inclination to make the most sincere and artful movie in which Adam Sandler will probably ever appear .',\n",
              "  'There is no substitute for on-screen chemistry , and when Friel pulls the strings that make Williams sink into melancholia , the reaction in Williams is as visceral as a gut punch .',\n",
              "  'That old adage about women being unknowable gets an exhilarating new interpretation in Morvern Callar .',\n",
              "  'A mix of gritty realism , crisp storytelling and radiant compassion that effortlessly draws you in .',\n",
              "  'After watching it , you can only love the players it brings to the fore for the gifted but no-nonsense human beings they are and for the still-inestimable contribution they have made to our shared history .',\n",
              "  'In his U.S. debut , Mr. Schnitzler proves himself a deft pace master and stylist .',\n",
              "  \"Rich in shadowy metaphor and as sharp as a samurai sword , Jiang Wen 's Devils on the Doorstep is a wartime farce in the alternately comic and gut-wrenching style of Joseph Heller or Kurt Vonnegut .\",\n",
              "  'Offers a clear-eyed chronicle of a female friendship that is more complex and honest than anything represented in a Hollywood film .',\n",
              "  'A winning comedy with its wry observations about long-lived friendships and the ways in which we all lose track of ourselves by trying to please others .',\n",
              "  \"Its cast full of caffeinated comedy performances more than make up for its logical loopholes , which fly by so fast there 's no time to think about them anyway .\",\n",
              "  \"Although it includes a fair share of dumb drug jokes and predictable slapstick , `` Orange County '' is far funnier than it would seem to have any right to be .\",\n",
              "  'A bright , inventive , thoroughly winning flight of revisionist fancy .',\n",
              "  \"Ozpetek 's effort has the scope and shape of an especially well-executed television movie .\",\n",
              "  'Affirms the gifts of all involved , starting with Spielberg and going right through the ranks of the players -- on-camera and off -- that he brings together .',\n",
              "  'A delightful little film that revels in its own simplicity , Mostly Martha will leave you with a smile on your face and a grumble in your stomach .',\n",
              "  'Makes one thing abundantly clear .',\n",
              "  \"American musical comedy as we know it would n't exist without the precedent of Yiddish theater , whose jolly , fun-for-fun 's - sake communal spirit goes to the essence of Broadway .\",\n",
              "  'Deepa Mehta provides an accessible introduction as well as some intelligent observations on the success of Bollywood in the Western world .',\n",
              "  'If anything , the film is doing something of a public service -- shedding light on a group of extremely talented musicians who might otherwise go unnoticed and underappreciated by music fans .',\n",
              "  'In addition to gluing you to the edge of your seat , Changing Lanes is also a film of freshness , imagination and insight .',\n",
              "  \"Pan Nalin 's exposition is beautiful and mysterious , and the interviews that follow , with the practitioners of this ancient Indian practice , are as subtle and as enigmatic .\",\n",
              "  'The mood , look and tone of the film fit the incredible storyline to a T.',\n",
              "  \"It 's crafty , energetic and smart -- the kid is sort of like a fourteen-year old Ferris Bueller .\",\n",
              "  'A work of extraordinary journalism , but it is also a work of deft and subtle poetry .',\n",
              "  \"It 's funny and human and really pretty damned wonderful , all at once .\",\n",
              "  'At 78 minutes it just zings along with vibrance and warmth .',\n",
              "  'A strangely stirring experience that finds warmth in the coldest environment and makes each crumb of emotional comfort feel like a 10-course banquet .',\n",
              "  '... plenty of warmth to go around , with music and laughter and the love of family .',\n",
              "  \"It 'll keep you wide awake and ... very tense .\",\n",
              "  'Could use a little more humanity , but it never lacks in eye-popping visuals .',\n",
              "  'These three films form a remarkably cohesive whole , both visually and thematically , through their consistently sensitive and often exciting treatment of an ignored people .',\n",
              "  \"A funny and well-contructed black comedy where the old adage `` be careful what you wish for '' is given a full workout .\",\n",
              "  'It reaffirms life as it looks in the face of death .',\n",
              "  'The film is reasonably entertaining , though it begins to drag two-thirds through , when the melodramatic aspects start to overtake the comedy .',\n",
              "  'This is more fascinating -- being real -- than anything seen on Jerry Springer .',\n",
              "  'A different movie -- sometimes tedious -- by a director many viewers would like to skip but film buffs should get to know .',\n",
              "  \"He does this so well you do n't have the slightest difficulty accepting him in the role .\",\n",
              "  'A vibrant whirlwind of love , family and all that goes with it , My Big Fat Greek Wedding is a non-stop funny feast of warmth , colour and cringe .',\n",
              "  'Thought-provoking and stylish , if also somewhat hermetic .',\n",
              "  \"Broomfield is energized by Volletta Wallace 's maternal fury , her fearlessness , and because of that , his film crackles .\",\n",
              "  \"While it has definite weaknesses -- like a rather unbelievable love interest and a meandering ending -- this '60s caper film is a riveting , brisk delight .\",\n",
              "  'Funny in a sick , twisted sort of way .',\n",
              "  \"If cinema had been around to capture the chaos of France in the 1790 's , one imagines the result would look like something like this .\",\n",
              "  \"It 's a talking head documentary , but a great one .\",\n",
              "  \"Ultimately engages less for its story of actorly existential despair than for its boundary-hopping formal innovations and glimpse into another kind of Chinese ` cultural revolution . '\",\n",
              "  '... a solid , well-formed satire .',\n",
              "  \"As part of Mr. Dong 's continuing exploration of homosexuality in America , Family Fundamentals is an earnest study in despair .\",\n",
              "  \"Most consumers of lo mein and General Tso 's chicken barely give a thought to the folks who prepare and deliver it , so , hopefully , this film will attach a human face to all those little steaming cartons .\",\n",
              "  'Hatosy ... portrays young Brendan with his usual intelligence and subtlety , not to mention a convincing brogue .',\n",
              "  \"The filmmakers ' eye for detail and the high standards of performance convey a strong sense of the girls ' environment .\",\n",
              "  'Uneven , self-conscious but often hilarious spoof .',\n",
              "  'Even bigger and more ambitious than the first installment , Spy Kids 2 looks as if it were made by a highly gifted 12-year-old instead of a grown man .',\n",
              "  'Alan and his fellow survivors are idiosyncratic enough to lift the movie above its playwriting 101 premise .',\n",
              "  'Fresh and raw like a blown-out vein , Narc takes a walking-dead , cop-flick subgenre and beats new life into it .',\n",
              "  'The premise of Jason X is silly but strangely believable .',\n",
              "  \"It 's a wise and powerful tale of race and culture forcefully told , with superb performances throughout .\",\n",
              "  'An awfully good , achingly human picture .',\n",
              "  'Definitely worth 95 minutes of your time .',\n",
              "  'The film jolts the laughs from the audience -- as if by cattle prod .',\n",
              "  'A sexy , surprising romance ... Idemoto and Kim make a gorgeous pair ... their scenes brim with sexual possibility and emotional danger .',\n",
              "  \"Toes the fine line between cheese and earnestness remarkably well ; everything is delivered with such conviction that it 's hard not to be carried away .\",\n",
              "  \"Whereas Oliver Stone 's conspiracy thriller JFK was long , intricate , star-studded and visually flashy , Interview with the Assassin draws its considerable power from simplicity .\",\n",
              "  'Funny , sexy , devastating and incurably romantic .',\n",
              "  \"Triple X is a double agent , and he 's one bad dude .\",\n",
              "  \"When you 've got the wildly popular Vin Diesel in the equation , it adds up to big box office bucks all but guaranteed .\",\n",
              "  'Very well-written and very well-acted .',\n",
              "  'A powerful and telling story that examines forbidden love , racial tension , and other issues that are as valid today as they were in the 1950s .',\n",
              "  \"You emerge dazed , confused as to whether you 've seen pornography or documentary .\",\n",
              "  \"It ai n't art , by a long shot , but unlike last year 's lame Musketeer , this Dumas adaptation entertains .\",\n",
              "  'likeable thanks to its cast , its cuisine and its quirky tunes .',\n",
              "  'Chilling in its objective portrait of dreary , lost twenty-first century America .',\n",
              "  'Highly recommended as an engrossing story about a horrifying historical event and the elements which contributed to it .',\n",
              "  \"... there 's enough cool fun here to warm the hearts of animation enthusiasts of all ages .\",\n",
              "  \"It manages to squeeze by on Angelina Jolie 's surprising flair for self-deprecating comedy .\",\n",
              "  'Secretary manages a neat trick , bundling the flowers of perversity , comedy and romance into a strangely tempting bouquet of a movie .',\n",
              "  \"Judith and Zaza 's extended bedroom sequence ... is so intimate and sensual and funny and psychologically self-revealing that it makes most of what passes for sex in the movies look like cheap hysterics .\",\n",
              "  'A knowing look at female friendship , spiked with raw urban humor .',\n",
              "  'As I settled into my World War II memories , I found myself strangely moved by even the corniest and most hackneyed contrivances .',\n",
              "  'The overall effect is awe and affection -- and a strange urge to get on a board and , uh , shred , dude .',\n",
              "  \"It 's that rare family movie -- genuine and sweet without relying on animation or dumb humor .\",\n",
              "  \"Confessions may not be a straightforward bio , nor does it offer much in the way of Barris ' motivations , but the film is an oddly fascinating depiction of an architect of pop culture .\",\n",
              "  'A special kind of movie , this melancholic film noir reminded me a lot of Memento ...',\n",
              "  \"Simple , poignant and leavened with humor , it 's a film that affirms the nourishing aspects of love and companionship .\",\n",
              "  'Together , Miller , Kuras and the actresses make Personal Velocity into an intricate , intimate and intelligent journey .',\n",
              "  'The wonder of Mostly Martha is the performance of Gedeck , who makes Martha enormously endearing .',\n",
              "  'With Notorious C.H.O. Cho proves she has the stuff to stand tall with Pryor , Carlin and Murphy .',\n",
              "  'Watching War Photographer , you come to believe that Nachtwey hates the wars he shows and empathizes with the victims he reveals .',\n",
              "  'Though it runs 163 minutes , Safe Conduct is anything but languorous .',\n",
              "  'A much better documentary -- more revealing , more emotional and more surprising -- than its pedestrian English title would have you believe .',\n",
              "  \"Notwithstanding my problem with the movie 's final half hour , I 'm going to recommend SECRETARY , based on the wonderful acting clinic put on by Spader and Gyllenhaal , and also the unique way Shainberg goes about telling what at heart is a sweet little girl -\",\n",
              "  'A well-crafted film that is all the more remarkable because it achieves its emotional power and moments of revelation with restraint and a delicate ambiguity .',\n",
              "  \"The film has the uncanny ability to right itself precisely when you think it 's in danger of going wrong .\",\n",
              "  \"My Big Fat Greek Wedding is that rare animal known as ' a perfect family film , ' because it 's about family .\",\n",
              "  \"would make an excellent companion piece to the similarly themed ` The French Lieutenant 's Woman . '\",\n",
              "  '... with the gifted Pearce on hand to keep things on semi-stable ground dramatically , this retooled Machine is ultimately effective enough at achieving the modest , crowd-pleasing goals it sets for itself .',\n",
              "  \"A disoriented but occasionally disarming saga packed with moments out of an Alice in Wonderland adventure , a stalker thriller , and a condensed season of TV 's Big Brother .\",\n",
              "  'Functions as both a revealing look at the collaborative process and a timely , tongue-in-cheek profile of the corporate circus that is the recording industry in the current climate of mergers and downsizing .',\n",
              "  'This odd , distant Portuguese import more or less borrows from Bad Lieutenant and Les Vampires , and comes up with a kind of art-house gay porn film .',\n",
              "  \"A photographic marvel of sorts , and it 's certainly an invaluable record of that special fishy community .\",\n",
              "  \"Although Olivier Assayas ' elegantly appointed period drama seems , at times , padded with incident in the way of a too-conscientious adaptation ... its three-hour running time plays closer to two .\",\n",
              "  \"Terry is a sort of geriatric Dirty Harry , which will please Eastwood 's loyal fans -- and suits the story , wherein our hero must ride roughshod over incompetent cops to get his man .\",\n",
              "  'A delightful entree in the tradition of food movies .',\n",
              "  \"An escapist confection that 's pure entertainment .\",\n",
              "  'Armed with a game supporting cast , from the pitch-perfect Forster to the always hilarious Meara and Levy , Like Mike shoots and scores , doing its namesake proud .',\n",
              "  'A decent-enough nail-biter that stands a good chance of being the big hit Franklin needs to stay afloat in Hollywood .',\n",
              "  'Begins like a docu-drama but builds its multi-character story with a flourish .',\n",
              "  'One of the most genuinely sweet films to come along in quite some time .',\n",
              "  \"After an uncertain start , Murder hits and generally sustains a higher plateau with Bullock 's memorable first interrogation of Gosling .\",\n",
              "  'The story ultimately takes hold and grips hard .',\n",
              "  'A bit of a downer and a little over-dramatic at times , but this is a beautiful film for people who like their romances to have that French realism .',\n",
              "  'An emotionally strong and politically potent piece of cinema .',\n",
              "  'Enticing and often funny documentary .',\n",
              "  \"A portrait of hell so shattering it 's impossible to shake .\",\n",
              "  \"Almodovar is an imaginative teacher of emotional intelligence in this engaging film about two men who discover what William James once called ` the gift of tears . '\",\n",
              "  'Tense , terrific , sweaty-palmed fun .',\n",
              "  \"Majidi 's direction has never been smoother or more confident .\",\n",
              "  'What a bewilderingly brilliant and entertaining movie this is .',\n",
              "  'Hard , endearing , caring , warm .',\n",
              "  'Bring tissues .',\n",
              "  \"A thriller with an edge -- which is to say that it does n't follow the stale , standard , connect-the-dots storyline which has become commonplace in movies that explore the seamy underbelly of the criminal world .\",\n",
              "  'Inherently caustic and oddly whimsical , the film chimes in on the grieving process and strangely draws the audience into the unexplainable pain and eccentricities that are attached to the concept of loss .',\n",
              "  \"It 's a square , sentimental drama that satisfies , as comfort food often can .\",\n",
              "  'Pure cinematic intoxication , a wildly inventive mixture of comedy and melodrama , tastelessness and swooning elegance .',\n",
              "  \"Ramsay is clearly extraordinarily talented , and based on three short films and two features , here 's betting her third feature will be something to behold .\",\n",
              "  'I was impressed by how many tit-for-tat retaliatory responses the filmmakers allow before pulling the plug on the conspirators and averting an American-Russian Armageddon .',\n",
              "  'A classy , sprightly spin on film .',\n",
              "  'A spiffy animated feature about an unruly adolescent boy who is yearning for adventure and a chance to prove his worth .',\n",
              "  'Devos and Cassel have tremendous chemistry -- their sexual and romantic tension , while never really vocalized , is palpable .',\n",
              "  'A moving , if uneven , success .',\n",
              "  'Wickedly funny , visually engrossing , never boring , this movie challenges us to think about the ways we consume pop culture .',\n",
              "  \"There 's plenty to impress about E.T.\",\n",
              "  \"A chronicle not only of one man 's quest to be president , but of how that man single-handedly turned a plane full of hard-bitten , cynical journalists into what was essentially , by campaign 's end , an extended publicity department .\",\n",
              "  \"Until it goes off the rails in its final 10 or 15 minutes , Wendigo , Larry Fessenden 's spooky new thriller , is a refreshingly smart and newfangled variation on several themes derived from far less sophisticated and knowing horror films .\",\n",
              "  \"Collateral Damage is trash , but it earns extra points by acting as if it were n't .\",\n",
              "  'A whole lot of fun and funny in the middle , though somewhat less hard-hitting at the start and finish .',\n",
              "  \"Maybe it is formula filmmaking , but there 's nothing wrong with that if the film is well-crafted and this one is .\",\n",
              "  \"The film is insightful about Kissinger 's background and history .\",\n",
              "  'An engrossing portrait of a man whose engaging manner and flamboyant style made him a truly larger-than-life character .',\n",
              "  \"A lot of the credit for the film 's winning tone must go to Grant , who has n't lost a bit of the dry humor that first made audiences on both sides of the Atlantic love him .\",\n",
              "  \"A journey that is as difficult for the audience to take as it is for the protagonist -- yet it 's potentially just as rewarding .\",\n",
              "  \"Ratliff 's two previous titles , Plutonium Circus and Purgatory County show his penchant for wry , contentious configurations , and this film is part of that delicate canon .\",\n",
              "  'From its invitingly upbeat overture to its pathos-filled but ultimately life-affirming finale , Martin is a masterfully conducted work .',\n",
              "  'Passions , obsessions , and loneliest dark spots are pushed to their most virtuous limits , lending the narrative an unusually surreal tone .',\n",
              "  'A comedy that swings and jostles to the rhythms of life .',\n",
              "  'At times Auto Focus feels so distant you might as well be watching it through a telescope .',\n",
              "  \"Yet in its own aloof , unreachable way it 's so fascinating you wo n't be able to look away for a second .\",\n",
              "  \"If you 're part of her targeted audience , you 'll cheer .\",\n",
              "  'As animation increasingly emphasizes the computer and the cool , this is a film that takes a stand in favor of tradition and warmth .',\n",
              "  'Blade II merges bits and pieces from fighting games , wire fu , horror movies , mystery , James Bond , wrestling , sci-fi and anime into one big bloody stew .',\n",
              "  'Instead of hitting the audience over the head with a moral , Schrader relies on subtle ironies and visual devices to convey point of view .',\n",
              "  'K-19 will not go down in the annals of cinema as one of the great submarine stories , but it is an engaging and exciting narrative of Man confronting the Demons of his own fear and paranoia .',\n",
              "  \"Contrived as this may sound , Mr. Rose 's updating works surprisingly well .\",\n",
              "  'A glib but bouncy bit of sixties-style slickness in which the hero might wind up caught but the audience gets pure escapism .',\n",
              "  \"You do n't need to be a hip-hop fan to appreciate Scratch , and that 's the mark of a documentary that works .\",\n",
              "  'Between bursts of automatic gunfire , the story offers a trenchant critique of capitalism .',\n",
              "  \"While the story 's undeniably hard to follow , Iwai 's gorgeous visuals seduce .\",\n",
              "  'If you can get past the taboo subject matter , it will be well worth your time .',\n",
              "  \"A lovely film ... elegant , witty and beneath a prim exterior unabashedly romantic ... hugely enjoyable in its own right though not really faithful to its source 's complexity .\",\n",
              "  'Scooby Doo is surely everything its fans are hoping it will be , and in that sense is a movie that deserves recommendation .',\n",
              "  'A solid cast , assured direction and complete lack of modern day irony .',\n",
              "  \"These characters are so well established that the gang feels comfortable with taking insane liberties and doing the goofiest stuff out of left field , and I 'm all for that .\",\n",
              "  'A sun-drenched masterpiece , part parlor game , part psychological case study , part droll social satire .',\n",
              "  'Worth a look as a curiosity .',\n",
              "  'You watch for that sense of openness , the little surprises .',\n",
              "  \"Director Peter Kosminsky gives these women a forum to demonstrate their acting ` chops ' and they take full advantage .\",\n",
              "  \"Schrader aims to present an unflinching look at one man 's downfall , brought about by his lack of self-awareness .\",\n",
              "  \"The Bourne Identity should n't be half as entertaining as it is , but director Doug Liman and his colleagues have managed to pack it with enough action to satisfy the boom-bam crowd without a huge sacrifice of character and mood .\",\n",
              "  'For VeggieTales fans , this is more appetizing than a side dish of asparagus .',\n",
              "  \"If you 're not a fan , it might be like trying to eat Brussels sprouts .\",\n",
              "  \"Remove Spider-Man the movie from its red herring surroundings and it 's apparent that this is one summer film that satisfies .\",\n",
              "  'A gorgeously strange movie , Heaven is deeply concerned with morality , but it refuses to spell things out for viewers .',\n",
              "  \"The Emperor 's Club , ruthless in its own placid way , finds one of our most conservative and hidebound movie-making traditions and gives it new texture , new relevance , new reality .\",\n",
              "  \"It 's truly awful and heartbreaking subject matter , but one whose lessons are well worth revisiting as many times as possible .\",\n",
              "  \"Though intrepid in exploring an attraction that crosses sexual identity , Ozpetek falls short in showing us Antonia 's true emotions ... But at the very least , His Secret Life will leave you thinking .\",\n",
              "  'There is little question that this is a serious work by an important director who has something new to say about how , in the flip-flop of courtship , we often reel in when we should be playing out .',\n",
              "  \"Gooding is the energetic frontman , and it 's hard to resist his enthusiasm , even if the filmmakers come up with nothing original in the way of slapstick sequences .\",\n",
              "  \"The otherwise good-naturedness of Mr. Deeds , with its embrace of sheer goofiness and cameos of less - than-likely New York celebrities ... certainly raises the film above anything Sandler 's been attached to before .\",\n",
              "  'The movie is brilliant , really .',\n",
              "  'It is philosophy , illustrated through everyday events .',\n",
              "  \"It 's stylishly directed with verve ...\",\n",
              "  'Gives an intriguing twist to the French coming-of-age genre .',\n",
              "  'Offers an interesting look at the rapidly changing face of Beijing .',\n",
              "  'A solid , psychological action film from Hong Kong .',\n",
              "  'Holm does his sly , intricate magic , and Iben Hjelje is entirely appealing as Pumpkin .',\n",
              "  'An enjoyable feel-good family comedy regardless of race .',\n",
              "  \"Features what is surely the funniest and most accurate depiction of writer 's block ever .\",\n",
              "  \"It would take a complete moron to foul up a screen adaptation of Oscar Wilde 's classic satire .\",\n",
              "  \"It 's bright , pristine style and bold colors make it as much fun as reading an oversized picture book before bedtime .\",\n",
              "  'Bravo for history rewritten , and for the uncompromising knowledge that the highest power of all is the power of love .',\n",
              "  '... a light , yet engrossing piece .',\n",
              "  'Lux , now in her eighties , does a great combination act as narrator , Jewish grandmother and subject -- taking us through a film that is part biography , part entertainment and part history .',\n",
              "  \"It 's a setup so easy it borders on facile , but keeping the film from cheap-shot mediocrity is its crack cast .\",\n",
              "  'Rife with the rueful , wry humor springing out of Yiddish culture and language .',\n",
              "  'A time machine , a journey back to your childhood , when cares melted away in the dark theater , and films had the ability to mesmerize , astonish and entertain .',\n",
              "  \"Rubbo 's humorously tendentious intervention into the who-wrote-Shakespeare controversy .\",\n",
              "  'Cantet beautifully illuminates what it means sometimes to be inside looking out , and at other times outside looking in .',\n",
              "  'K-19 : The Widowmaker is a great yarn .',\n",
              "  \"It 's as raw and action-packed an experience as a ringside seat at a tough-man contest .\",\n",
              "  'Evokes the frustration , the awkwardness and the euphoria of growing up , without relying on the usual tropes .',\n",
              "  'Visually striking and viscerally repellent .',\n",
              "  'Overcomes its visual hideousness with a sharp script and strong performances .',\n",
              "  'Astonishingly skillful and moving ... it could become a historically significant work as well as a masterfully made one .',\n",
              "  'Beautifully crafted and cooly unsettling ... recreates the atmosphere of the crime expertly .',\n",
              "  'The year 2002 has conjured up more coming-of-age stories than seem possible , but Take Care of My Cat emerges as the very best of them .',\n",
              "  'The sight of the spaceship on the launching pad is duly impressive in IMAX dimensions , as are shots of the astronauts floating in their cabins .',\n",
              "  'Time is a beautiful film to watch , an interesting and at times captivating take on loss and loneliness .',\n",
              "  'An intriguing look at the French film industry during the German occupation ; its most delightful moments come when various characters express their quirky inner selves .',\n",
              "  \"A fine documentary can be distinguished from a mediocre one by the better film 's ability to make its subject interesting to those who are n't part of its supposed target audience .\",\n",
              "  \"Judging by those standards , ` Scratch ' is a pretty decent little documentary .\",\n",
              "  'A diverse and astonishingly articulate cast of Palestinian and Israeli children .',\n",
              "  'Slight but enjoyable documentary .',\n",
              "  'Unfolds in a low-key , organic way that encourages you to accept it as life and go with its flow .',\n",
              "  'A beguiling evocation of the quality that keeps Dickens evergreen : the exuberant openness with which he expresses our most basic emotions .',\n",
              "  'The heat of the moment prevails .',\n",
              "  'It cooks Conduct in a low , smoky and inviting sizzle .',\n",
              "  'A riveting story well told .',\n",
              "  'A compelling pre-WWII drama with vivid characters and a warm , moving message .',\n",
              "  'The stars may be college kids , but the subject matter is as adult as you can get : the temptations of the flesh are unleashed by a slightly crazed , overtly determined young woman and a one-night swim turns into an ocean of trouble .',\n",
              "  'Pretty good little movie .',\n",
              "  'By turns touching , raucously amusing , uncomfortable , and , yes , even sexy , Never Again is a welcome and heartwarming addition to the romantic comedy genre .',\n",
              "  \"If you have n't seen the film lately , you may be surprised at the variety of tones in Spielberg 's work .\",\n",
              "  'Much of it is funny , but there are also some startling , surrealistic moments ...',\n",
              "  'An entertaining mix of period drama and flat-out farce that should please history fans .',\n",
              "  \"Canada 's arctic light shines bright on this frozen tundra soap opera that breathes extraordinary life into the private existence of the Inuit people .\",\n",
              "  'Alternately hilarious and sad , aggravating and soulful , scathing and joyous .',\n",
              "  \"It 's a masterpeice .\",\n",
              "  \"The film 's messages of tolerance and diversity are n't particularly original , but one ca n't help but be drawn in by the sympathetic characters .\",\n",
              "  \"Though it lacks the utter authority of a genre gem , there 's a certain robustness to this engaging mix of love and bloodletting .\",\n",
              "  \"A conventional , but well-crafted film about a historic legal battle in Ireland over a man 's right to raise his own children .\",\n",
              "  \"Yes , it 's as good as you remember .\",\n",
              "  'In fact , even better .',\n",
              "  'Hartley adds enough quirky and satirical touches in the screenplay to keep the film entertaining .',\n",
              "  'An uncomfortable movie , suffocating and sometimes almost senseless , The Grey Zone does have a center , though a morbid one .',\n",
              "  'A stirring road movie .',\n",
              "  'One of the best films I have ever seen , constantly pulling the rug from underneath us , seeing things from new sides , plunging deeper , getting more intense .',\n",
              "  'Insanely hilarious !',\n",
              "  \"I have n't laughed that hard in years !\",\n",
              "  'These people are really going to love The Piano Teacher .',\n",
              "  \"-- but it makes for one of the most purely enjoyable and satisfying evenings at the movies I 've had in a while .\",\n",
              "  \"If `` Lilo & Stitch '' is n't the most edgy piece of Disney animation to hit the silver screen , then this first film to use a watercolor background since `` Dumbo '' certainly ranks as the most original in years .\",\n",
              "  'See Scratch for the history , see Scratch for the music , see Scratch for a lesson in scratching , but , most of all , see it for the passion .',\n",
              "  \"... `` Bowling for Columbine '' remains a disquieting and thought-provoking film ...\",\n",
              "  \"Talk to Her is so darned assured , we have absolutely no idea who the main characters are until the film is well under way -- and yet it 's hard to stop watching .\",\n",
              "  \"Star\\\\/producer Salma Hayek and director Julie Taymor have infused Frida with a visual style unique and inherent to the titular character 's paintings and in the process created a masterful work of art of their own .\",\n",
              "  'A truly wonderful tale combined with stunning animation .',\n",
              "  'A low-key labor of love that strikes a very resonant chord .',\n",
              "  'An average kid-empowerment fantasy with slightly above-average brains .',\n",
              "  \"Confessions is n't always coherent , but it 's sharply comic and surprisingly touching , so hold the gong .\",\n",
              "  'A sentimental but entirely irresistible portrait of three aging sisters .',\n",
              "  \"White Oleander may leave you rolling your eyes in the dark , but that does n't mean you wo n't like looking at it .\",\n",
              "  'In painting an unabashedly romantic picture of a nation whose songs spring directly from the lives of the people , the movie exalts the Marxian dream of honest working folk , with little to show for their labor , living harmoniously , joined in song .',\n",
              "  \"The most brilliant work in this genre since the 1984 uncut version of Sergio Leone 's flawed but staggering Once Upon a Time in America .\",\n",
              "  'It looks closely , insightfully at fragile , complex relationships .',\n",
              "  'A fine effort , an interesting topic , some intriguing characters and a sad ending .',\n",
              "  \"Certainly the big finish was n't something Galinsky and Hawley could have planned for ... but part of being a good documentarian is being there when the rope snaps .\",\n",
              "  'It must be the end of the world : the best film so far this year is a franchise sequel starring Wesley Snipes .',\n",
              "  'There are moments of hilarity to be had .',\n",
              "  'A hypnotic portrait of this sad , compulsive life .',\n",
              "  'One of the best films of the year with its exquisite acting , inventive screenplay , mesmerizing music , and many inimitable scenes of tenderness , loss , discontent , and yearning .',\n",
              "  'Return to Never Land is reliable , standard Disney animated fare , with enough creative energy and wit to entertain all ages .',\n",
              "  \"Michael Moore 's latest documentary about America 's thirst for violence is his best film yet ...\",\n",
              "  'Like a Tarantino movie with heart , Alias Betty is richly detailed , deftly executed and utterly absorbing .',\n",
              "  'Marvelously entertaining and deliriously joyous documentary .',\n",
              "  'A brisk , reverent , and subtly different sequel .',\n",
              "  'A movie I loved on first sight and , even more important , love in remembrance .',\n",
              "  'Deserves a place of honor next to Nanook as a landmark in film history .',\n",
              "  'Murderous Maids pulls no punches in its depiction of the lives of the Papin sister and the events that led to their notorious rise to infamy ...',\n",
              "  'This is an undeniably intriguing film from an adventurous young talent who finds his inspiration on the fringes of the American underground .',\n",
              "  'But it does have one saving grace .',\n",
              "  \"This is lightweight filmmaking , to be sure , but it 's pleasant enough -- and oozing with attractive men .\",\n",
              "  'At its most basic , this cartoon adventure is that wind-in-the-hair exhilarating .',\n",
              "  \"Fans of critics ' darling band Wilco will marvel at the sometimes murky , always brooding look of I Am Trying to Break Your Heart .\",\n",
              "  'The film presents visceral and dangerously honest revelations about the men and machines behind the curtains of our planet .',\n",
              "  'Fulford-Wierzbicki ... deftly captures the wise-beyond-her-years teen .',\n",
              "  'A wild ride juiced with enough energy and excitement for at least three films .',\n",
              "  \"It 's a cool event for the whole family .\",\n",
              "  'Maybe not a classic , but a movie the kids will want to see over and over again .',\n",
              "  'The movie is not as terrible as the synergistic impulse that created it .',\n",
              "  'Less an examination of neo-Nazism than a probe into the nature of faith itself .',\n",
              "  \"A moving and weighty depiction of one family 's attempts to heal after the death of a child .\",\n",
              "  \"I do n't think most of the people who loved the 1989 Paradiso will prefer this new version .\",\n",
              "  'Diggs and Lathan are among the chief reasons Brown Sugar is such a sweet and sexy film .',\n",
              "  'Entirely suspenseful , extremely well-paced and ultimately ... dare I say , entertaining !',\n",
              "  'The riveting performances by the incredibly flexible cast make Love a joy to behold .',\n",
              "  \"Terrific as Nadia , a Russian mail-order bride who comes to America speaking not a word of English , it 's Kidman who holds the film together with a supremely kittenish performance that gradually accumulates more layers .\",\n",
              "  'With an unflappable air of decadent urbanity , Everett remains a perfect Wildean actor , and a relaxed Firth displays impeccable comic skill .',\n",
              "  \"The re-release of Ron Howard 's Apollo 13 in the IMAX format proves absolutely that really , really , really good things can come in enormous packages .\",\n",
              "  'Very well written and directed with brutal honesty and respect for its audience .',\n",
              "  'Wonderful fencing scenes and an exciting plot make this an eminently engrossing film .',\n",
              "  \"It 's pretty linear and only makeup-deep , but Bogdanovich ties it together with efficiency and an affection for the period .\",\n",
              "  \"A surprisingly charming and even witty match for the best of Hollywood 's comic-book adaptations .\",\n",
              "  'This is a superior horror flick .',\n",
              "  'Adaptation is simply brilliant .',\n",
              "  'Smart and alert , Thirteen Conversations About One Thing is a small gem .',\n",
              "  'The pleasure of Read My Lips is like seeing a series of perfect black pearls clicking together to form a string .',\n",
              "  \"We 're drawn in by the dark luster .\",\n",
              "  'A haunting tale of murder and mayhem .',\n",
              "  'I love the opening scenes of a wintry New York City in 1899 .',\n",
              "  \"Cinematic poetry showcases the city 's old-world charm before machines change nearly everything .\",\n",
              "  \"It 's hard to imagine anyone managing to steal a movie not only from charismatic rising star Jake Gyllenhaal but also from accomplished Oscar winners Susan Sarandon , Dustin Hoffman and Holly Hunter , yet newcomer Ellen Pompeo pulls off the feat with aplomb .\",\n",
              "  'One of the best rock documentaries ever .',\n",
              "  'Wilco is a phenomenal band with such an engrossing story that will capture the minds and hearts of many .',\n",
              "  'Ian Holm conquers France as an earthy Napoleon',\n",
              "  'Offers big , fat , dumb laughs that may make you hate yourself for giving in .',\n",
              "  'Directors Brett Morgen and Nanette Burstein have put together a bold biographical fantasia .',\n",
              "  'The subtitled costume drama is set in a remote African empire before cell phones , guns , and the internal combustion engine , but the politics that thump through it are as timely as tomorrow .',\n",
              "  'A tremendous piece of work .',\n",
              "  'A delightful , if minor , pastry of a movie .',\n",
              "  'While obviously aimed at kids , The Country Bears ... should keep parents amused with its low groan-to-guffaw ratio .',\n",
              "  'LaBute masterfully balances both Traditional or Modern stories together in a manner that one never overwhelms the other .',\n",
              "  'Something for everyone .',\n",
              "  \"Irwin is so earnest that it 's hard to resist his pleas to spare wildlife and respect their environs .\",\n",
              "  \"Leigh succeeds in delivering a dramatic slap in the face that 's simultaneously painful and refreshing .\",\n",
              "  'Not about scares but a mood in which an ominous , pervasive , and unknown threat lurks just below the proceedings and adds an almost constant mindset of suspense .',\n",
              "  \"` Film aficionados can not help but love Cinema Paradiso , whether the original version or new Director 's Cut . '\",\n",
              "  \"It 's one thing to read about or rail against the ongoing - and unprecedented - construction project going on over our heads .\",\n",
              "  \"Contradicts everything we 've come to expect from movies nowadays .\",\n",
              "  'Instead of simply handling conventional material in a conventional way , Secretary takes the most unexpected material and handles it in the most unexpected way .',\n",
              "  'Could I have been more geeked when I heard that Apollo 13 was going to be released in IMAX format ?',\n",
              "  'Murderous Maids has a lot going for it , not least the brilliant performances by Testud ... and Parmentier .',\n",
              "  \"Filmmaker Stacy Peralta has a flashy editing style that does n't always jell with Sean Penn 's monotone narration , but he respects the material without sentimentalizing it .\",\n",
              "  \"There are a couple of things that elevate `` Glory '' above most of its ilk , most notably the mere presence of Duvall .\",\n",
              "  \"It 's light on the chills and heavy on the atmospheric weirdness , and there are moments of jaw-droppingly odd behavior -- yet I found it weirdly appealing .\",\n",
              "  'A breezy blend of art , history , esoteric musings and philosophy .',\n",
              "  'Kids will love its fantasy and adventure , and grownups should appreciate its whimsical humor .',\n",
              "  'Leaping from one arresting image to another , Songs from the Second Floor has all the enjoyable randomness of a very lively dream and so manages to be compelling , amusing and unsettling at the same time .',\n",
              "  'The performances are uniformly good .',\n",
              "  \"She 's all-powerful , a voice for a pop-cyber culture that feeds on her Bjorkness .\",\n",
              "  \"It 's a perfect show of respect to just one of those underrated professionals who deserve but rarely receive it .\",\n",
              "  ...),\n",
              " (['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['negative'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ['positive'],\n",
              "  ...))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "KO_XRoi0cHKS",
        "colab": {}
      },
      "source": [
        "vocab.fit(iterator.get_instances(data_type=\"train\")[1])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "WqrLgM3wcHKU",
        "outputId": "9885bc33-7f88-4053-b11e-e2ad9374c7d7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "list(vocab.items())"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[('positive', 0), ('negative', 1)]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "07UYYS-bcHKW",
        "outputId": "612b524c-a5a6-4b9e-b149-0ddd380dfff4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "vocab([\"positive\", \"positive\", \"negative\"])"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[0, 0, 1]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "UJgn639-cHKZ",
        "outputId": "8ead8862-3256-4634-c699-6ee131a930d2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "vocab([0, 0, 1])"
      ],
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['positive', 'positive', 'negative']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "CFpd2ciccHKc"
      },
      "source": [
        "**One-hotter**\n",
        "\n",
        "**DOCS:** http://docs.deeppavlov.ai/en/latest/apiref/models/preprocessors.html"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "DiXLIj4EcHKc",
        "colab": {}
      },
      "source": [
        "from deeppavlov.models.preprocessors.one_hotter import OneHotter"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "6EtTb1VgcHKe",
        "colab": {}
      },
      "source": [
        "one_hotter = OneHotter(depth=vocab.len, \n",
        "                       single_vector=True  # means we want to have one vector per sample\n",
        "                      )"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "YTh-O4uvcHKg",
        "outputId": "f8eed7c6-be1b-4cd8-fcce-e87f18e15c16",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 70
        }
      },
      "source": [
        "one_hotter(vocab([\"positive\", \"positive\", \"negative\"]))"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[array([1., 0.], dtype=float32),\n",
              " array([1., 0.], dtype=float32),\n",
              " array([0., 1.], dtype=float32)]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "RgjLcf5McHKh"
      },
      "source": [
        "**Converting from probability to labels**\n",
        "\n",
        "Neural model not only accepts one-hot classes representation but also returns for every sample vector of probability distribution of classes. Therefore, we need to use some component to convert probability ditribution to label indices. \n",
        "\n",
        "`Proba2Labels` component supports three different model:\n",
        "* if `max_proba` is true, returns indices of the highest probabilities,\n",
        "* if `confident_threshold` is given, returns indices with probabiltiies higher than threshold,\n",
        "* if `top_n` is given, returns `top_n` indices with highest probabilities.\n",
        "\n",
        "**DOCS:** http://docs.deeppavlov.ai/en/latest/apiref/models/preprocessors.html"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "jwZ_zKZYcHKi",
        "colab": {}
      },
      "source": [
        "from deeppavlov.models.classifiers.proba2labels import Proba2Labels\n",
        "\n",
        "prob2labels = Proba2Labels(max_proba=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n4eKuCvqYkqZ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "a84377d0-1bef-4464-f1f3-b8a363317764"
      },
      "source": [
        "vocab.len"
      ],
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "zVsUXstmcHKk",
        "outputId": "ecfe12fe-d483-49b1-a0ff-2fe5551644dc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "prob2labels([[0.6, 0.4], \n",
        "             [0.2, 0.8],\n",
        "             [0.1, 0.9]])"
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[[0], [1], [1]]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1qCNgjSVGOMP",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "e5c66e10-0939-4359-f9bc-b0620d7b284d"
      },
      "source": [
        "vocab(prob2labels([[0.6, 0.4], \n",
        "                   [0.2, 0.8],\n",
        "                   [0.1, 0.9]]))"
      ],
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[['positive'], ['negative'], ['negative']]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "CnUUkw_AcHKo"
      },
      "source": [
        "## Classifier\n",
        "\n",
        "DeepPavlov contains several classification components: sklearn classifiers, NNs on Keras, BERT classifier on tensorflow. This tutorial demonstrates how to build Convolutional neural network classifier on Keras. \n",
        "\n",
        "`KerasClassificationModel` is a class building Keras classifier where network architecture is built in a separate class method. \n",
        "\n",
        "**DOCS:** http://docs.deeppavlov.ai/en/latest/apiref/models/classifiers.html"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "XgGTTfgqcHKo",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "97bc4ef2-30dc-4a56-a47a-e73c1137f042"
      },
      "source": [
        "from keras.layers import Input, Dense, Activation, Dropout, Flatten, GlobalMaxPooling1D\n",
        "from keras import Model\n",
        "\n",
        "from deeppavlov.models.classifiers.keras_classification_model import KerasClassificationModel\n",
        "from deeppavlov.metrics.accuracy import sets_accuracy"
      ],
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using TensorFlow backend.\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "tM6IoNGqcHKp",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "e5520967-3419-41c5-b985-988937efa8e4"
      },
      "source": [
        "model = KerasClassificationModel(\n",
        "    filters_cnn=256,\n",
        "    kernel_sizes_cnn=[3,5,7],\n",
        "    dropout_rate=0.2,\n",
        "    dense_size=100,\n",
        "    save_path=\"./cnn_model_v0\", \n",
        "    load_path=\"./cnn_model_v0\", \n",
        "    embedding_size=embedder.dim,\n",
        "    n_classes=vocab.len,\n",
        "    model_name=\"cnn_model\",  # HERE we put our new network-method name\n",
        "    optimizer=\"Adam\",\n",
        "    learning_rate=0.001,\n",
        "    learning_rate_decay=0.001,\n",
        "    loss=\"categorical_crossentropy\")"
      ],
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2019-07-01 02:34:28.477 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 244: [initializing `KerasClassificationModel` from scratch as cnn_model]\n",
            "2019-07-01 02:34:29.196 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 134: Model was successfully initialized!\n",
            "Model summary:\n",
            "__________________________________________________________________________________________________\n",
            "Layer (type)                    Output Shape         Param #     Connected to                     \n",
            "==================================================================================================\n",
            "input_1 (InputLayer)            (None, None, 100)    0                                            \n",
            "__________________________________________________________________________________________________\n",
            "conv1d_1 (Conv1D)               (None, None, 256)    77056       input_1[0][0]                    \n",
            "__________________________________________________________________________________________________\n",
            "conv1d_2 (Conv1D)               (None, None, 256)    128256      input_1[0][0]                    \n",
            "__________________________________________________________________________________________________\n",
            "conv1d_3 (Conv1D)               (None, None, 256)    179456      input_1[0][0]                    \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_1 (BatchNor (None, None, 256)    1024        conv1d_1[0][0]                   \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_2 (BatchNor (None, None, 256)    1024        conv1d_2[0][0]                   \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_3 (BatchNor (None, None, 256)    1024        conv1d_3[0][0]                   \n",
            "__________________________________________________________________________________________________\n",
            "activation_1 (Activation)       (None, None, 256)    0           batch_normalization_1[0][0]      \n",
            "__________________________________________________________________________________________________\n",
            "activation_2 (Activation)       (None, None, 256)    0           batch_normalization_2[0][0]      \n",
            "__________________________________________________________________________________________________\n",
            "activation_3 (Activation)       (None, None, 256)    0           batch_normalization_3[0][0]      \n",
            "__________________________________________________________________________________________________\n",
            "global_max_pooling1d_1 (GlobalM (None, 256)          0           activation_1[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "global_max_pooling1d_2 (GlobalM (None, 256)          0           activation_2[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "global_max_pooling1d_3 (GlobalM (None, 256)          0           activation_3[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "concatenate_1 (Concatenate)     (None, 768)          0           global_max_pooling1d_1[0][0]     \n",
            "                                                                 global_max_pooling1d_2[0][0]     \n",
            "                                                                 global_max_pooling1d_3[0][0]     \n",
            "__________________________________________________________________________________________________\n",
            "dropout_1 (Dropout)             (None, 768)          0           concatenate_1[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "dense_1 (Dense)                 (None, 100)          76900       dropout_1[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_4 (BatchNor (None, 100)          400         dense_1[0][0]                    \n",
            "__________________________________________________________________________________________________\n",
            "activation_4 (Activation)       (None, 100)          0           batch_normalization_4[0][0]      \n",
            "__________________________________________________________________________________________________\n",
            "dropout_2 (Dropout)             (None, 100)          0           activation_4[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "dense_2 (Dense)                 (None, 2)            202         dropout_2[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_5 (BatchNor (None, 2)            8           dense_2[0][0]                    \n",
            "__________________________________________________________________________________________________\n",
            "activation_5 (Activation)       (None, 2)            0           batch_normalization_5[0][0]      \n",
            "==================================================================================================\n",
            "Total params: 465,350\n",
            "Trainable params: 463,610\n",
            "Non-trainable params: 1,740\n",
            "__________________________________________________________________________________________________\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "7ITs_wpucHKq",
        "scrolled": true,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 267
        },
        "outputId": "251d5b68-fe35-4052-ee69-a3961e011118"
      },
      "source": [
        "# Method `get_instances` returns all the samples of particular data field\n",
        "x_valid, y_valid = iterator.get_instances(data_type=\"valid\")\n",
        "# You need to save model only when validation score is higher than previous one.\n",
        "# This variable will contain the highest accuracy score\n",
        "best_score = 0.\n",
        "patience = 2\n",
        "impatience = 0\n",
        "\n",
        "# let's train for 10 epochs\n",
        "for ep in range(10):\n",
        "    \n",
        "    for x, y in iterator.gen_batches(batch_size=64, \n",
        "                                     data_type=\"train\", shuffle=True):\n",
        "        x_embed = embedder(tokenizer(preprocessor(x)))\n",
        "        y_onehot = one_hotter(vocab(y))\n",
        "        model.train_on_batch(x_embed, y_onehot)\n",
        "        \n",
        "    y_valid_pred = model(embedder(tokenizer(preprocessor(x_valid))))\n",
        "    score = sets_accuracy(y_valid, vocab(prob2labels(y_valid_pred)))\n",
        "    print(\"Epochs done: {}. Valid Accuracy: {}\".format(ep + 1, score))\n",
        "    if score > best_score:\n",
        "        model.save()\n",
        "        print(\"New best score. Saving model.\")\n",
        "        best_score = score    \n",
        "        impatience = 0\n",
        "    else:\n",
        "      impatience += 1\n",
        "      if impatience == patience:\n",
        "        print(\"Out of patience. Stop training.\")\n",
        "        break"
      ],
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2019-07-01 02:35:35.782 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 373: [saving model to /content/cnn_model_v0_opt.json]\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Epochs done: 1. Valid Accuracy: 0.7915151515151515\n",
            "New best score. Saving model.\n",
            "Epochs done: 2. Valid Accuracy: 0.7890909090909091\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "2019-07-01 02:36:57.485 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 373: [saving model to /content/cnn_model_v0_opt.json]\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Epochs done: 3. Valid Accuracy: 0.7951515151515152\n",
            "New best score. Saving model.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "2019-07-01 02:37:38.513 INFO in 'deeppavlov.models.classifiers.keras_classification_model'['keras_classification_model'] at line 373: [saving model to /content/cnn_model_v0_opt.json]\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Epochs done: 4. Valid Accuracy: 0.8315151515151515\n",
            "New best score. Saving model.\n",
            "Epochs done: 5. Valid Accuracy: 0.7915151515151515\n",
            "Epochs done: 6. Valid Accuracy: 0.8024242424242424\n",
            "Out of patience. Stop training.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "6mXMUtrpcHKt",
        "outputId": "582733fe-888b-4095-fe61-aaafbe9be575",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 87
        }
      },
      "source": [
        "# Let's look into obtained resulting outputs\n",
        "print(\"Text sample: {}\".format(x_valid[0]))\n",
        "print(\"True label: {}\".format(y_valid[0]))\n",
        "print(\"Predicted probability distribution: {}\".format(dict(zip(vocab.keys(), \n",
        "                                                               y_valid_pred[0]))))\n",
        "print(\"Predicted label: {}\".format(vocab(prob2labels(y_valid_pred))[0]))"
      ],
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Text sample: It 's a lovely film with lovely performances by Buy and Accorsi .\n",
            "True label: ['positive']\n",
            "Predicted probability distribution: {'positive': 0.8794663548469543, 'negative': 0.048511240631341934}\n",
            "Predicted label: ['positive']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "cXMGov74cHKv"
      },
      "source": [
        "# Fine-grained classification\n",
        "\n",
        "Fine-grained labelled dataset corresponds to multi-class classification task with 5 classes.\n",
        "Still this classification is not multi-label, so you do not need to change anything from binary classifiaction except of network or training parameters.\n",
        "\n",
        "The **TASK** is to build from scratch fine-grained classifier."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "u55H8o1VcHKv",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "UESbbxL7cHKz",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}