Merge branch 'develop'

CHANGELOG.md

@@ -6,6 +6,21 @@ Most recent releases are shown at the top. Each release shows:
 - **Changed**: Additional parameters, changes to inputs or outputs, etc
 - **Fixed**: Bug fixes that don't change documented behaviour
 
+## 0.21.4 (2020-09-24)
+
+### New:
+- N/A
+
+### Changed
+- Changed installation instructions in `README.md` to reflect that using *ktrain* with TensorFlow 2.1 will require downgrading `transformers` to 3.1.0.
+- updated requirements with `keras_bert>=0.86.0` due to TensorFlow 2.3 error with older versions of `keras_bert`
+- In  `lr_find` and `lr_plot`, check for TF 2.2 or 2.3 and make necessary adjustments due to TF bug 41174.
+
+### Fixed:
+- fixed typos in `__all__` in `text` and graph` modules (PR #250)
+- fixed Chinese language translation based on on name-changes of models with `zh` as source language
+
+
 ## 0.21.3 (2020-09-08)
 
 ### New:

FAQ.md

@@ -231,7 +231,8 @@ If your machine has a GPU (which is needed for larger models), you'll need to pe
   and copy the two files in `C:\Users\<your_user_name>\Miniconda3\envs\kt\Lib\site-packages\pywin32_system32` to `C:\Windows\System32`.
 - If you experience SSL certificate problems with either `pip` or `conda`, run `conda config --set ssl_verify false` and 
 replace all `pip` comands above with `pip --trusted-host pypi.org --trusted-host files.pythonhosted.org`.
-- In the instructions above, we are installing TensorFlow 2.3. Note that there is a bug in TensorFlow 2.2 and 2.3 that affects the *Learning-Rate-Finder* [that will not be fixed until TensorFlow 2.4](https://github.com/tensorflow/tensorflow/issues/41174#issuecomment-656330268).  You can avoid the bug by using `tensorflow==2.1.0` and `transformers==3.1.0`. 
+- In the instructions above, we are installing TensorFlow 2.3. Note that there is a bug in both TensorFlow 2.3 and 2.2 affecting the *Learning-Rate-Finder* [that will not be fixed until TensorFlow 2.4](https://github.com/tensorflow/tensorflow/issues/41174#issuecomment-656330268).  The bug causes the learning-rate-finder to complete all epochs even after loss has diverged (i.e., no automatic-stopping).
+- If using `tensorflow<=2.1`, you must also downgrade **transformers** to `transformers==3.1` to avoid errors.
 - We have selected Python 3.7 in STEP 4 above with `python=3.7` for illustration purposes, but Python 3.8 is default if removed. 
 
 #### Running an Example

examples/text/language_translation_example.ipynb

@@ -251,7 +251,7 @@
    "source": [
     "## The `Translator` Class for Translating to and from Many Languages\n",
     "\n",
-    "For translations **from** and **to** other languages, `text.Translator`instances can be used. `Translator` instances accept as input a pretrained model from [Helsinki-NLP](https://huggingface.co/models?search=Helsinki-NLP%2Fopus-mt).  For instance, to translate Chinese to German, one can use the [Helsinki-NLP/opus-mt-ZH-de ](https://huggingface.co/Helsinki-NLP/opus-mt-ZH-de) model:"
+    "For translations **from** and **to** other languages, `text.Translator`instances can be used. `Translator` instances accept as input a pretrained model from [Helsinki-NLP](https://huggingface.co/models?search=Helsinki-NLP%2Fopus-mt).  For instance, to translate Chinese to German, one can use the [Helsinki-NLP/opus-mt-zh-de ](https://huggingface.co/Helsinki-NLP/opus-mt-zh-de) model:"
    ]
   },
   {
@@ -269,8 +269,8 @@
     }
    ],
    "source": [
-    "translator = text.Translator(model_name='Helsinki-NLP/opus-mt-ZH-de')\n",
-    "src_text = '''大流行对世界经济造成了严重破坏。但是，截至2020年6月，美国股票市场持续上涨。'''\n",
+    "translator = text.Translator(model_name='Helsinki-NLP/opus-mt-zh-de')\n",
+    "src_text = '''冠状病毒大流行对世界经济造成了严重破坏。但是，截至2020年6月，美国股市继续上涨。'''\n",
     "print(translator.translate(src_text))"
    ]
   },

ktrain/core.py

@@ -517,6 +517,13 @@ def lr_find(self, start_lr=1e-7, lr_mult=1.01, max_epochs=None, class_weight=Non
         Returns:
             None
         """
+        # dep_fix: bug in TF 2.2 and 2.3
+        if version.parse(tf.__version__) > version.parse('2.1') and version.parse(tf.__version__) < version.parse('2.4'):
+            if max_epochs is None:
+                warnings.warn('Due to a bug in TensorFlow 2.2 and 2.3, the max_epochs argument is temporarily required. Please re-run with max_epochs. \n' +\
+                              'More info: https://github.com/tensorflow/tensorflow/issues/41174#issuecomment-656330268')
+                return
+
 
         U.vprint('simulating training for different learning rates... this may take a few moments...',
                 verbose=verbose)
@@ -621,6 +628,10 @@ def lr_plot(self, n_skip_beginning=10, n_skip_end=5, suggest=False, return_fig=F
           matplotlib.figure.Figure if return_fig else None
           
         """
+        # dep_fix: bug in TF 2.2 and 2.3
+        if version.parse(tf.__version__) > version.parse('2.1') and version.parse(tf.__version__) < version.parse('2.4'):
+            if n_skip_end == 5: n_skip_end=10
+
         if self.lr_finder is None or not self.lr_finder.find_called(): raise ValueError('Please call lr_find first.')
         return self.lr_finder.plot_loss(n_skip_beginning=n_skip_beginning,
                                         n_skip_end=n_skip_end, suggest=suggest, return_fig=return_fig)

ktrain/graph/__init__.py

@@ -2,11 +2,11 @@
 from .data import *
 #from .predictor import *
 __all__ = [
-           'graph_nodes_from_csv'
-           'graph_links_from_csv'
+           'graph_nodes_from_csv',
+           'graph_links_from_csv',
            'print_node_classifiers',
            'print_link_predictors',
-           'node_classifier'
-           'link_predictor'
+           'graph_node_classifier',
+           'graph_link_predictor'
            ]
 

ktrain/tests/test_textclassification.py

@@ -209,7 +209,7 @@ def test_bigru(self):
 
         # test training results
         self.assertAlmostEqual(max(hist.history['lr']), lr)
-        self.assertGreater(max(hist.history[VAL_ACC_NAME]), 0.9)
+        self.assertGreater(max(hist.history[VAL_ACC_NAME]), 0.89)
         self.assertAlmostEqual(max(hist.history['momentum']), 0.95)
         self.assertAlmostEqual(min(hist.history['momentum']), 0.85)
 

ktrain/text/__init__.py

@@ -26,7 +26,7 @@
            'print_sequence_taggers',
            'get_topic_model',
            'Transformer',
-           'TranformerEmbedding',
+           'TransformerEmbedding',
            'shallownlp',
            'TransformerSummarizer',
            'ZeroShotClassifier',

ktrain/text/translation/core.py

@@ -124,9 +124,11 @@ def __init__(self, src_lang=None, device=None):
             self.translators.append(Translator(model_name='Helsinki-NLP/opus-mt-af-en', device=device))
         elif src_lang in ['es', 'fr', 'it', 'pt']:
             self.translators.append(Translator(model_name='Helsinki-NLP/opus-mt-ROMANCE-en', device=device))
-        elif src_lang == 'zh': # could not find zh->en model, so currently doing two-step translation to English via German
-            self.translators.append(Translator(model_name='Helsinki-NLP/opus-mt-ZH-de', device=device))
-            self.translators.append(Translator(model_name='Helsinki-NLP/opus-mt-de-en', device=device))
+        #elif src_lang == 'zh': # could not find zh->en model, so currently doing two-step translation to English via German
+            #self.translators.append(Translator(model_name='Helsinki-NLP/opus-mt-ZH-de', device=device))
+            #self.translators.append(Translator(model_name='Helsinki-NLP/opus-mt-de-en', device=device))
+        elif src_lang == 'zh':
+            self.translators.append(Translator(model_name='Helsinki-NLP/opus-mt-zh-en', device=device))
         else:
             raise ValueError('lang:%s is currently not supported.' % (src_lang))
 

ktrain/version.py

@@ -1,2 +1,2 @@
 __all__ = ['__version__']
-__version__ = '0.21.3'
+__version__ = '0.21.4'

setup.py

@@ -31,7 +31,7 @@
           'matplotlib >= 3.0.0',
           'pandas >= 1.0.1',
           'fastprogress >= 0.1.21',
-          'keras_bert>=0.81.0',
+          'keras_bert>=0.86.0', # support for TF 2.3
           'requests',
           'joblib',
           'langdetect',

tutorials/tutorial-04-text-classification.ipynb

@@ -95,7 +95,8 @@
     "\n",
     "For our text classification example, we will again classifiy IMDb movie reviews as either positive or negative.  However, instead of using the pre-processed version of the dataset pre-packaged with Keras, we will use the original (or raw) *aclImdb* dataset.  The dataset can be downloaded from [here](http://ai.stanford.edu/~amaas/data/sentiment/). Set the ```DATADIR``` variable to the location of the extracted *aclImdb* folder.\n",
     "\n",
-    "In the cell below, note that we supplied `preprocess_mode='standard'` to the data-loading function (which is the default).  For pretrained models like BERT, the dataset must be preprocessed in a specific way.  If you are planning to use BERT for text classification, you should replace this argument with `preprocess_mode='bert'`. Since we will not be using BERT in this example, we leave it as `preprocess_mode='standard'`.  See [this notebook](https://github.com/amaiya/ktrain/blob/master/examples/text/IMDb-BERT.ipynb) for an example of how to use BERT for text classification in *ktrain*."
+    "In the cell below, note that we supplied `preprocess_mode='standard'` to the data-loading function (which is the default).  For pretrained models like BERT and DistilBERT, the dataset must be preprocessed in a specific way.  If you are planning to use BERT for text classification, you should replace this argument with `preprocess_mode='bert'`. Since we will not be using BERT in this example, we leave it as `preprocess_mode='standard'`.  See [this notebook](https://github.com/amaiya/ktrain/blob/master/examples/text/IMDb-BERT.ipynb) for an example of how to use BERT for text classification in *ktrain*. There is also a [DistilBERT example notebook](https://github.com/amaiya/ktrain/blob/master/examples/text/20newsgroup-distilbert.ipynb).   \n",
+    "**NOTE:**  If using `preprocess_mode='bert'` or `preprocess_mode='distilbert'`, an English pretrained model is used for English, a Chinese pretrained model is used for Chinese, and a multilingual pretrained model is used for all other languages. For more flexibility in choosing the model used, you can use the alternative [Transformer API for text classification](https://github.com/amaiya/ktrain/blob/master/tutorials/tutorial-A3-hugging_face_transformers.ipynb) in *ktrain*.  "
    ]
   },
   {
@@ -155,9 +156,11 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Having loaded the data, we will now create a text classification model.  The `print_text_classifier` function prints some available models.  (One can also use the `Transformer` API in *ktrain* to access an even larger library of Hugging Face Transformer models like RoBERTa and XLNet. See [this tutorial](https://github.com/amaiya/ktrain/blob/master/tutorials/tutorial-A3-hugging_face_transformers.ipynb) for more information on this.)   \n",
+    "Having loaded the data, we will now create a text classification model.  The `print_text_classifier` function prints some available models.  The model selected should be consistent with the `preprocess_mode` selected above. \n",
     "\n",
-    "In this example, the ```text_classifier``` function returns a [neural implementation of NBSVM](https://medium.com/@asmaiya/a-neural-implementation-of-nbsvm-in-keras-d4ef8c96cb7c), which is a strong baseline that can outperform more complex neural architectures. It may take a few moments to return as it builds a document-term matrix from the input data we provide it. The ```text_classifier```  function expects `trn` to be a preprocessed training set returned from the `texts_from*` function above. In this case where we have used `preprocess_mode='standard'`, `trn` is a numpy array with each document represented as fixed-size sequence of word IDs."
+    "(As mentioned above, one can also use the alternative `Transformer` API for text classification in *ktrain* to access an even larger library of Hugging Face Transformer models like RoBERTa and XLNet. See [this tutorial](https://github.com/amaiya/ktrain/blob/master/tutorials/tutorial-A3-hugging_face_transformers.ipynb) for more information on this.)   \n",
+    "\n",
+    "In this example, the `text_classifier` function will return a [neural implementation of NBSVM](https://medium.com/@asmaiya/a-neural-implementation-of-nbsvm-in-keras-d4ef8c96cb7c), which is a strong baseline that can outperform more complex neural architectures. It may take a few moments to return as it builds a document-term matrix from the input data we provide it. The ```text_classifier```  function expects `trn` to be a preprocessed training set returned from the `texts_from*` function above. In this case where we have used `preprocess_mode='standard'`, `trn` is a numpy array with each document represented as fixed-size sequence of word IDs."
    ]
   },
   {

tutorials/tutorial-05-learning_from_unlabeled_text_data.ipynb

@@ -2415,7 +2415,7 @@
     "\n",
     "In the previous section, given a set of seed documents, we scored **new** documents based on similarity.  Here, we will reverse this process. Given a **new** document, we will find (or recommend) documents that are semantically similar to it from the 20newsgroup corpus.\n",
     "\n",
-    "We must first train the recommender."
+    "We must first train the recommender. The `train_recommender` method trains a [Nearest Neighbors model](https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.NearestNeighbors.html#sklearn.neighbors.NearestNeighbors) that can be used to perform **semantic searches** and generate **document recommendations** on your dataset."
    ]
   },
   {
@@ -2431,7 +2431,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Now, let's create some text about space exploration and recommend the top newsgroup posts similar to this text."
+    "Now, let's create some text about space exploration and recommend the top newsgroup posts similar to this text. "
    ]
   },
   {
@@ -2451,7 +2451,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Here is the top recommended 20newsgroup post based on semantic similarity to the text above."
+    "Here is the top recommended 20newsgroup post based on semantic similarity to the text above. This can be considered a **semantic text search** as the `recommend` method will return documents that are semantically-related to the supplied text."
    ]
   },
   {