From 76816fe5b76fe1ddc3e68d1386e806e8920753e2 Mon Sep 17 00:00:00 2001 From: anakin87 <44616784+anakin87@users.noreply.github.com> Date: Tue, 9 Aug 2022 17:50:09 +0200 Subject: [PATCH 01/18] first draft for tutorial extension --- tutorials/Tutorial14_Query_Classifier.ipynb | 349 ++++++++++++++------ 1 file changed, 242 insertions(+), 107 deletions(-) diff --git a/tutorials/Tutorial14_Query_Classifier.ipynb b/tutorials/Tutorial14_Query_Classifier.ipynb index 17f1430264..de075dfa50 100644 --- a/tutorials/Tutorial14_Query_Classifier.ipynb +++ b/tutorials/Tutorial14_Query_Classifier.ipynb @@ -69,11 +69,11 @@ "cell_type": "code", "execution_count": null, "metadata": { + "collapsed": true, "id": "CjA5n5lMN-gd", "pycharm": { "name": "#%%\n" - }, - "collapsed": true + } }, "outputs": [], "source": [ @@ -91,37 +91,37 @@ }, { "cell_type": "markdown", + "metadata": { + "collapsed": false, + "pycharm": { + "name": "#%% md\n" + } + }, "source": [ - "## Logging\n", + "### Logging\n", "\n", "We configure how logging messages should be displayed and which log level should be used before importing Haystack.\n", "Example log message:\n", "INFO - haystack.utils.preprocessing - Converting data/tutorial1/218_Olenna_Tyrell.txt\n", "Default log level in basicConfig is WARNING so the explicit parameter is not necessary but can be changed easily:" - ], - "metadata": { - "collapsed": false, - "pycharm": { - "name": "#%% md\n" - } - } + ] }, { "cell_type": "code", "execution_count": null, + "metadata": { + "collapsed": false, + "pycharm": { + "name": "#%%\n" + } + }, "outputs": [], "source": [ "import logging\n", "\n", "logging.basicConfig(format=\"%(levelname)s - %(name)s - %(message)s\", level=logging.WARNING)\n", "logging.getLogger(\"haystack\").setLevel(logging.INFO)" - ], - "metadata": { - "collapsed": false, - "pycharm": { - "name": "#%%\n" - } - } + ] }, { "cell_type": "markdown", @@ -138,8 +138,8 @@ "cell_type": "code", "execution_count": null, "metadata": { - "id": "XhPMEqBzxA8V", - "collapsed": true + "collapsed": true, + "id": "XhPMEqBzxA8V" }, "outputs": [], "source": [ @@ -151,39 +151,44 @@ }, { "cell_type": "markdown", - "source": [ - "Now let's feed some queries into this query classifier. We'll test with one keyword query, one interrogative query, and one statement query. Notice that we don't use any punctuation, such as question marks; this illustrates that the classifier doesn't need punctuation in order to make the right decision." - ], "metadata": { "id": "1NHjy9aa9FKx" - } + }, + "source": [ + "Now let's feed some queries into this query classifier. We'll test with one keyword query, one interrogative query, and one statement query. Notice that we don't use any punctuation, such as question marks; this illustrates that the classifier doesn't need punctuation in order to make the right decision." + ] }, { "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "Ks7qdULR8J13" + }, + "outputs": [], "source": [ "queries = [\n", " \"Arya Stark father\", # Keyword Query\n", " \"Who was the father of Arya Stark\", # Interrogative Query\n", " \"Lord Eddard was the father of Arya Stark\", # Statement Query\n", "]" - ], - "metadata": { - "id": "Ks7qdULR8J13" - }, - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", - "source": [ - "We can see below what our classifier does with these queries: \"Arya Stark father\" is rightly determined to be a keyword query and is sent to branch 2, while both the interrogative query \"Who was the father of Arya Stark\" and the statement query \"Lord Eddard was the father of Arya Stark\" are correctly labeled as non-keyword queries, and are thus shipped off to branch 1." - ], "metadata": { "id": "UbKlyXcNj-nx" - } + }, + "source": [ + "We can see below what our classifier does with these queries: \"Arya Stark father\" is rightly determined to be a keyword query and is sent to branch 2, while both the interrogative query \"Who was the father of Arya Stark\" and the statement query \"Lord Eddard was the father of Arya Stark\" are correctly labeled as non-keyword queries, and are thus shipped off to branch 1." + ] }, { "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "NYROmSHnE4zp" + }, + "outputs": [], "source": [ "import pandas as pd\n", "\n", @@ -196,28 +201,23 @@ " k_vs_qs_results[\"Class\"].append(\"Question/Statement\" if result[1] == \"output_1\" else \"Keyword\")\n", "\n", "pd.DataFrame.from_dict(k_vs_qs_results)" - ], - "metadata": { - "id": "NYROmSHnE4zp" - }, - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", - "source": [ - "Next we will illustrate a **question vs. statement** `SklearnQueryClassifier`. We define our classifier below; notice that this time we have to explicitly specify the model and vectorizer, since the default for an `SklearnQueryClassifier` (and a `TransformersQueryClassifier`) is keyword vs. question/statement classification." - ], "metadata": { "id": "VyMZzRVHlG5O" - } + }, + "source": [ + "Next we will illustrate a **question vs. statement** `SklearnQueryClassifier`. We define our classifier below; notice that this time we have to explicitly specify the model and vectorizer, since the default for an `SklearnQueryClassifier` (and a `TransformersQueryClassifier`) is keyword vs. question/statement classification." + ] }, { "cell_type": "code", "execution_count": null, "metadata": { - "id": "l4eH3SSaxZ0O", - "collapsed": true + "collapsed": true, + "id": "l4eH3SSaxZ0O" }, "outputs": [], "source": [ @@ -234,15 +234,20 @@ }, { "cell_type": "markdown", - "source": [ - "We will test this classifier on the two question/statement queries from the last go-round:" - ], "metadata": { "id": "zdAY1CUYnTFa" - } + }, + "source": [ + "We will test this classifier on the two question/statement queries from the last go-round:" + ] }, { "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "1ZULHEBVmqq2" + }, + "outputs": [], "source": [ "queries = [\n", " \"Who was the father of Arya Stark\", # Interrogative Query\n", @@ -258,35 +263,35 @@ " q_vs_s_results[\"Class\"].append(\"Question\" if result[1] == \"output_1\" else \"Statement\")\n", "\n", "pd.DataFrame.from_dict(q_vs_s_results)" - ], - "metadata": { - "id": "1ZULHEBVmqq2" - }, - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", - "source": [ - "And as we see, the question \"Who was the father of Arya Stark\" is sent to branch 1, while the statement \"Lord Eddard was the father of Arya Stark\" is sent to branch 2, so we can have our pipeline treat statements and questions differently." - ], "metadata": { "id": "Fk2kpvQR6Fa0" - } + }, + "source": [ + "And as we see, the question \"Who was the father of Arya Stark\" is sent to branch 1, while the statement \"Lord Eddard was the father of Arya Stark\" is sent to branch 2, so we can have our pipeline treat statements and questions differently." + ] }, { "cell_type": "markdown", + "metadata": { + "id": "eEwDIq9KXXke" + }, "source": [ "### Using Query Classifiers in a Pipeline\n", "\n", "Now let's see how we can use query classifiers in a question-answering (QA) pipeline. We start by initiating Elasticsearch:" - ], - "metadata": { - "id": "eEwDIq9KXXke" - } + ] }, { "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "fCLtLItU5aWl" + }, + "outputs": [], "source": [ "# In Colab / No Docker environments: Start Elasticsearch from source\n", "! wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.9.2-linux-x86_64.tar.gz -q\n", @@ -301,12 +306,7 @@ ")\n", "# wait until ES has started\n", "! sleep 30" - ], - "metadata": { - "id": "fCLtLItU5aWl" - }, - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", @@ -324,11 +324,11 @@ "cell_type": "code", "execution_count": null, "metadata": { + "collapsed": true, "id": "Ig7dgfdHN-gg", "pycharm": { "name": "#%%\n" - }, - "collapsed": true + } }, "outputs": [], "source": [ @@ -361,19 +361,25 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "CbAgZ2MZn2qm" + }, "source": [ "#### Pipelines with Keyword vs. Question/Statement Classification\n", "\n", "Our first illustration will be a simple retriever-reader QA pipeline, but the choice of which retriever we use will depend on the type of query received: **keyword** queries will use a sparse **`BM25Retriever`**, while **question/statement** queries will use the more accurate but also more computationally expensive **`EmbeddingRetriever`**.\n", "\n", "We start by initializing our retrievers and reader:" - ], - "metadata": { - "id": "CbAgZ2MZn2qm" - } + ] }, { "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true, + "id": "m7zOPYQ-Ylep" + }, + "outputs": [], "source": [ "# Initialize sparse retriever for keyword queries\n", "bm25_retriever = BM25Retriever(document_store=document_store)\n", @@ -385,13 +391,7 @@ "document_store.update_embeddings(embedding_retriever, update_existing_embeddings=False)\n", "\n", "reader = FARMReader(model_name_or_path=\"deepset/roberta-base-squad2\")" - ], - "metadata": { - "id": "m7zOPYQ-Ylep", - "collapsed": true - }, - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", @@ -406,11 +406,11 @@ "cell_type": "code", "execution_count": null, "metadata": { + "collapsed": true, "id": "Sz-oZ5eJN-gl", "pycharm": { "name": "#%%\n" - }, - "collapsed": true + } }, "outputs": [], "source": [ @@ -429,19 +429,19 @@ }, { "cell_type": "markdown", - "source": [ - "Below we can see some results from this choice in branching structure: the keyword query \"arya stark father\" and the question query \"Who is the father of Arya Stark?\" generate noticeably different results, a distinction that is likely due to the use of different retrievers for keyword vs. question/statement queries." - ], "metadata": { "id": "imqRRCGTwQav" - } + }, + "source": [ + "Below we can see some results from this choice in branching structure: the keyword query \"arya stark father\" and the question query \"Who is the father of Arya Stark?\" generate noticeably different results, a distinction that is likely due to the use of different retrievers for keyword vs. question/statement queries." + ] }, { "cell_type": "code", "execution_count": null, "metadata": { - "id": "fP6Cpcb-o0HK", - "collapsed": true + "collapsed": true, + "id": "fP6Cpcb-o0HK" }, "outputs": [], "source": [ @@ -532,8 +532,8 @@ "cell_type": "code", "execution_count": null, "metadata": { - "id": "BIisEJrzDr-9", - "collapsed": true + "collapsed": true, + "id": "BIisEJrzDr-9" }, "outputs": [], "source": [ @@ -553,15 +553,20 @@ }, { "cell_type": "markdown", - "source": [ - "And below we see the results of this pipeline: with a question query like \"Who is the father of Arya Stark?\" we get back answers returned by a reader, but with a statement query like \"Arya Stark was the daughter of a Lord\" we just get back documents returned by a retriever." - ], "metadata": { "id": "QU1B6JQEDrol" - } + }, + "source": [ + "And below we see the results of this pipeline: with a question query like \"Who is the father of Arya Stark?\" we get back answers returned by a reader, but with a statement query like \"Arya Stark was the daughter of a Lord\" we just get back documents returned by a retriever." + ] }, { "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "HIjgs5k7C6CN" + }, + "outputs": [], "source": [ "# Useful for framing headers\n", "equal_line = \"=\" * 30\n", @@ -576,12 +581,139 @@ "res_2 = transformer_question_classifier.run(query=\"Arya Stark was the daughter of a Lord.\")\n", "print(f\"\\n\\n{equal_line}\\nSTATEMENT QUERY RESULTS\\n{equal_line}\")\n", "print_documents(res_2)" - ], - "metadata": { - "id": "HIjgs5k7C6CN" - }, + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Other use cases for Query Classifiers: custom classification models and zero shot classification\n", + "\n", + "`TransformersQueryClassifier` is very flexible and also supports other possibilities for classifying queries, including loading a custom classification model from Transformers Hub or using zero shot classification." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Using custom classification models\n", + "We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose a appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.### Using zero-shot-classification" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from haystack.nodes import TransformersQueryClassifier\n", + "\n", + "# remember to compile a list with the exact model labels\n", + "# the first provided label will corresponds to output_1, the second label to output_2, and so on.\n", + "labels = [\"LABEL_0\", \"LABEL_1\", \"LABEL_2\"]\n", + "\n", + "sentiment_query_classifier = TransformersQueryClassifier(\n", + " model_name_or_path=\"cardiffnlp/twitter-roberta-base-sentiment\",\n", + " use_gpu=True,\n", + " task=\"text-classification\",\n", + " labels=labels,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "queries = [\n", + " \"What's the answer?\", # neutral query\n", + " \"Would you be so lovely to tell me the answer?\", # positive query\n", + " \"Can you give me the damn right answer for once??\", # negative query\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "\n", + "sent_results = {\"Query\": [], \"Output Branch\": [], \"Class\": []}\n", + "\n", + "for query in queries:\n", + " result = sentiment_query_classifier.run(query=query)\n", + " sent_results[\"Query\"].append(query)\n", + " sent_results[\"Output Branch\"].append(result[1])\n", + " if result[1] == \"output_1\":\n", + " sent_results[\"Class\"].append(\"negative\")\n", + " elif result[1] == \"output_2\":\n", + " sent_results[\"Class\"].append(\"neutral\")\n", + " elif result[1] == \"output_3\":\n", + " sent_results[\"Class\"].append(\"positive\")\n", + "\n", + "pd.DataFrame.from_dict(sent_results)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Using zero shot classification\n", + "It is also possible to perform zero shot classifcation, by providing a suitable base transformer model and **choosing** the classes that the model should predict.\n", + "For example, we may be interested in whether the user query is related to music or cinema." + ] + }, + { + "cell_type": "code", "execution_count": null, - "outputs": [] + "metadata": {}, + "outputs": [], + "source": [ + "# in zero-shot-classification, the labels can be freely chosen\n", + "labels = [\"music\", \"cinema\"]\n", + "\n", + "query_classifier = TransformersQueryClassifier(\n", + " model_name_or_path=\"typeform/distilbert-base-uncased-mnli\",\n", + " use_gpu=True,\n", + " task=\"zero-shot-classification\",\n", + " labels=labels,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "queries = [\n", + " \"In which films does John Travolta appear?\", # query about cinema\n", + " \"What is the Rolling Stones first album?\", # query about music\n", + " \"Who was Sergio Leone?\", # query about cinema\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "\n", + "query_classification_results = {\"Query\": [], \"Output Branch\": [], \"Class\": []}\n", + "\n", + "for query in queries:\n", + " result = query_classifier.run(query=query)\n", + " query_classification_results[\"Query\"].append(query)\n", + " query_classification_results[\"Output Branch\"].append(result[1])\n", + " query_classification_results[\"Class\"].append(\"music\" if result[1] == \"output_1\" else \"cinema\")\n", + "\n", + "pd.DataFrame.from_dict(query_classification_results)" + ] }, { "cell_type": "markdown", @@ -615,11 +747,9 @@ "name": "Tutorial14_Query_Classifier.ipynb", "provenance": [] }, - "interpreter": { - "hash": "01829e1eb67c4f5275a41f9336c92adbb77a108c8fc957dfe99d03e96dd1f349" - }, "kernelspec": { - "display_name": "Python 3.9.5 64-bit ('venv': venv)", + "display_name": "Python 3.7.13 ('venv': venv)", + "language": "python", "name": "python3" }, "language_info": { @@ -632,9 +762,14 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.5" + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "8db71796417881f0bc38f90aea8b7586b2d6898630e76f69c59e615595ffff43" + } } }, "nbformat": 4, "nbformat_minor": 0 -} \ No newline at end of file +} From 480a4c78705ed5340c42224467204930d06bda7c Mon Sep 17 00:00:00 2001 From: anakin87 <44616784+anakin87@users.noreply.github.com> Date: Tue, 9 Aug 2022 18:06:15 +0200 Subject: [PATCH 02/18] forgotten markdown --- docs/_src/tutorials/tutorials/14.md | 95 ++++++++++++++++++++++++++++- 1 file changed, 94 insertions(+), 1 deletion(-) diff --git a/docs/_src/tutorials/tutorials/14.md b/docs/_src/tutorials/tutorials/14.md index 649697f92d..740c62dec2 100644 --- a/docs/_src/tutorials/tutorials/14.md +++ b/docs/_src/tutorials/tutorials/14.md @@ -56,7 +56,7 @@ Next we make sure the latest version of Haystack is installed: !pip install pygraphviz ``` -## Logging +### Logging We configure how logging messages should be displayed and which log level should be used before importing Haystack. Example log message: @@ -339,6 +339,99 @@ print(f"\n\n{equal_line}\nSTATEMENT QUERY RESULTS\n{equal_line}") print_documents(res_2) ``` +### Other use cases for Query Classifiers: custom classification models and zero shot classification + +`TransformersQueryClassifier` is very flexible and also supports other possibilities for classifying queries, including loading a custom classification model from Transformers Hub or using zero shot classification. + +#### Using custom classification models +We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose a appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.### Using zero-shot-classification + + +```python +from haystack.nodes import TransformersQueryClassifier + +# remember to compile a list with the exact model labels +# the first provided label will corresponds to output_1, the second label to output_2, and so on. +labels = ["LABEL_0", "LABEL_1", "LABEL_2"] + +sentiment_query_classifier = TransformersQueryClassifier( + model_name_or_path="cardiffnlp/twitter-roberta-base-sentiment", + use_gpu=True, + task="text-classification", + labels=labels, +) +``` + + +```python +queries = [ + "What's the answer?", # neutral query + "Would you be so lovely to tell me the answer?", # positive query + "Can you give me the damn right answer for once??", # negative query +] +``` + + +```python +import pandas as pd + +sent_results = {"Query": [], "Output Branch": [], "Class": []} + +for query in queries: + result = sentiment_query_classifier.run(query=query) + sent_results["Query"].append(query) + sent_results["Output Branch"].append(result[1]) + if result[1] == "output_1": + sent_results["Class"].append("negative") + elif result[1] == "output_2": + sent_results["Class"].append("neutral") + elif result[1] == "output_3": + sent_results["Class"].append("positive") + +pd.DataFrame.from_dict(sent_results) +``` + +#### Using zero shot classification +It is also possible to perform zero shot classifcation, by providing a suitable base transformer model and **choosing** the classes that the model should predict. +For example, we may be interested in whether the user query is related to music or cinema. + + +```python +# in zero-shot-classification, the labels can be freely chosen +labels = ["music", "cinema"] + +query_classifier = TransformersQueryClassifier( + model_name_or_path="typeform/distilbert-base-uncased-mnli", + use_gpu=True, + task="zero-shot-classification", + labels=labels, +) +``` + + +```python +queries = [ + "In which films does John Travolta appear?", # query about cinema + "What is the Rolling Stones first album?", # query about music + "Who was Sergio Leone?", # query about cinema +] +``` + + +```python +import pandas as pd + +query_classification_results = {"Query": [], "Output Branch": [], "Class": []} + +for query in queries: + result = query_classifier.run(query=query) + query_classification_results["Query"].append(query) + query_classification_results["Output Branch"].append(result[1]) + query_classification_results["Class"].append("music" if result[1] == "output_1" else "cinema") + +pd.DataFrame.from_dict(query_classification_results) +``` + ## About us This [Haystack](https://github.com/deepset-ai/haystack/) notebook was made with love by [deepset](https://deepset.ai/) in Berlin, Germany From 455396f7d6bc0519d5f12206fc5f4e2115a2e40c Mon Sep 17 00:00:00 2001 From: anakin87 <44616784+anakin87@users.noreply.github.com> Date: Wed, 10 Aug 2022 09:53:57 +0200 Subject: [PATCH 03/18] improved tutorial --- docs/_src/tutorials/tutorials/14.md | 14 ++++++++++++-- tutorials/Tutorial14_Query_Classifier.ipynb | 16 +++++++++++++--- 2 files changed, 25 insertions(+), 5 deletions(-) diff --git a/docs/_src/tutorials/tutorials/14.md b/docs/_src/tutorials/tutorials/14.md index 740c62dec2..a8ec89f16e 100644 --- a/docs/_src/tutorials/tutorials/14.md +++ b/docs/_src/tutorials/tutorials/14.md @@ -341,10 +341,18 @@ print_documents(res_2) ### Other use cases for Query Classifiers: custom classification models and zero shot classification -`TransformersQueryClassifier` is very flexible and also supports other possibilities for classifying queries, including loading a custom classification model from Transformers Hub or using zero shot classification. +`TransformersQueryClassifier` is very flexible and also supports other possibilities for classifying queries. +For example, we may be interested in detecting the sentiment or classifying the topics. This can be done by loading a custom classification model from Transformers Hub or by using zero shot classification. + +#### Custom classification model vs zero shot classification +- traditional text classification models are trained to predict one of a few "hard-coded" classes and required a dedicated training dataset. In Transformers Hub you can find many pre-trained models, maybe even related to your domain of interest. +- zero-shot classification is very versatile: choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories. #### Using custom classification models -We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose a appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.### Using zero-shot-classification +We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose a appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment. + +*In this case, the `labels` parameter must contain a list with the exact model labels. +The first provided label will corresponds to output_1, the second label to output_2, and so on.* ```python @@ -395,6 +403,8 @@ pd.DataFrame.from_dict(sent_results) It is also possible to perform zero shot classifcation, by providing a suitable base transformer model and **choosing** the classes that the model should predict. For example, we may be interested in whether the user query is related to music or cinema. +*In this case, the `labels` parameter is a list containing the candidate classes.* + ```python # in zero-shot-classification, the labels can be freely chosen diff --git a/tutorials/Tutorial14_Query_Classifier.ipynb b/tutorials/Tutorial14_Query_Classifier.ipynb index de075dfa50..0290c1fa0b 100644 --- a/tutorials/Tutorial14_Query_Classifier.ipynb +++ b/tutorials/Tutorial14_Query_Classifier.ipynb @@ -589,7 +589,12 @@ "source": [ "### Other use cases for Query Classifiers: custom classification models and zero shot classification\n", "\n", - "`TransformersQueryClassifier` is very flexible and also supports other possibilities for classifying queries, including loading a custom classification model from Transformers Hub or using zero shot classification." + "`TransformersQueryClassifier` is very flexible and also supports other possibilities for classifying queries.\n", + "For example, we may be interested in detecting the sentiment or classifying the topics. This can be done by loading a custom classification model from Transformers Hub or by using zero shot classification.\n", + "\n", + "#### Custom classification model vs zero shot classification\n", + "- traditional text classification models are trained to predict one of a few \"hard-coded\" classes and required a dedicated training dataset. In Transformers Hub you can find many pre-trained models, maybe even related to your domain of interest.\n", + "- zero-shot classification is very versatile: choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories." ] }, { @@ -597,7 +602,10 @@ "metadata": {}, "source": [ "#### Using custom classification models\n", - "We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose a appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.### Using zero-shot-classification" + "We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose a appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.\n", + "\n", + "*In this case, the `labels` parameter must contain a list with the exact model labels.\n", + "The first provided label will corresponds to output_1, the second label to output_2, and so on.*" ] }, { @@ -663,7 +671,9 @@ "source": [ "#### Using zero shot classification\n", "It is also possible to perform zero shot classifcation, by providing a suitable base transformer model and **choosing** the classes that the model should predict.\n", - "For example, we may be interested in whether the user query is related to music or cinema." + "For example, we may be interested in whether the user query is related to music or cinema.\n", + "\n", + "*In this case, the `labels` parameter is a list containing the candidate classes.*" ] }, { From 49ddbc5f670fb8bfd73f19bb6c1502dc1bae5ead Mon Sep 17 00:00:00 2001 From: Stefano Fiorucci <44616784+anakin87@users.noreply.github.com> Date: Wed, 10 Aug 2022 11:43:26 +0200 Subject: [PATCH 04/18] Apply suggestions from code review Co-authored-by: Agnieszka Marzec <97166305+agnieszka-m@users.noreply.github.com> --- tutorials/Tutorial14_Query_Classifier.ipynb | 36 ++++++++++----------- 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/tutorials/Tutorial14_Query_Classifier.ipynb b/tutorials/Tutorial14_Query_Classifier.ipynb index 0290c1fa0b..592b96751e 100644 --- a/tutorials/Tutorial14_Query_Classifier.ipynb +++ b/tutorials/Tutorial14_Query_Classifier.ipynb @@ -155,7 +155,7 @@ "id": "1NHjy9aa9FKx" }, "source": [ - "Now let's feed some queries into this query classifier. We'll test with one keyword query, one interrogative query, and one statement query. Notice that we don't use any punctuation, such as question marks; this illustrates that the classifier doesn't need punctuation in order to make the right decision." + "Now let's feed some queries into this query classifier. We'll test with one keyword query, one interrogative query, and one statement query. Note that we don't need to use any punctuation, such as question marks, for the query classifier to make the right decision." ] }, { @@ -179,7 +179,7 @@ "id": "UbKlyXcNj-nx" }, "source": [ - "We can see below what our classifier does with these queries: \"Arya Stark father\" is rightly determined to be a keyword query and is sent to branch 2, while both the interrogative query \"Who was the father of Arya Stark\" and the statement query \"Lord Eddard was the father of Arya Stark\" are correctly labeled as non-keyword queries, and are thus shipped off to branch 1." + "Below, you can see what the classifier does with these queries: it correctly determines that \"Arya Stark father\" is a keyword query and sends it to branch 2. It also correctly classifies both the interrogative query \"Who was the father of Arya Stark\" and the statement query \"Lord Eddard was the father of Arya Stark\" as non-keyword queries, and sends them to branch 1." ] }, { @@ -209,7 +209,7 @@ "id": "VyMZzRVHlG5O" }, "source": [ - "Next we will illustrate a **question vs. statement** `SklearnQueryClassifier`. We define our classifier below; notice that this time we have to explicitly specify the model and vectorizer, since the default for an `SklearnQueryClassifier` (and a `TransformersQueryClassifier`) is keyword vs. question/statement classification." + "Next, we will illustrate a **question vs. statement** `SklearnQueryClassifier`. We define our classifier below. Note that this time we have to explicitly specify the model and vectorizer since the default for a `SklearnQueryClassifier` (and a `TransformersQueryClassifier`) is keyword vs. question/statement classification." ] }, { @@ -271,7 +271,7 @@ "id": "Fk2kpvQR6Fa0" }, "source": [ - "And as we see, the question \"Who was the father of Arya Stark\" is sent to branch 1, while the statement \"Lord Eddard was the father of Arya Stark\" is sent to branch 2, so we can have our pipeline treat statements and questions differently." + "And as we see, the question \"Who was the father of Arya Stark\" is sent to branch 1, while the statement \"Lord Eddard was the father of Arya Stark\" is sent to branch 2. This means we can have our pipeline treat statements and questions differently." ] }, { @@ -433,7 +433,7 @@ "id": "imqRRCGTwQav" }, "source": [ - "Below we can see some results from this choice in branching structure: the keyword query \"arya stark father\" and the question query \"Who is the father of Arya Stark?\" generate noticeably different results, a distinction that is likely due to the use of different retrievers for keyword vs. question/statement queries." + "Below, we can see how this choice affects the branching structure: the keyword query \"arya stark father\" and the question query \"Who is the father of Arya Stark?\" generate noticeably different results, a distinction that is likely due to the use of different retrievers for keyword vs. question/statement queries." ] }, { @@ -557,7 +557,7 @@ "id": "QU1B6JQEDrol" }, "source": [ - "And below we see the results of this pipeline: with a question query like \"Who is the father of Arya Stark?\" we get back answers returned by a reader, but with a statement query like \"Arya Stark was the daughter of a Lord\" we just get back documents returned by a retriever." + "And here are the results of this pipeline: with a question query like \"Who is the father of Arya Stark?\", we obtain answers from a reader, and with a statement query like \"Arya Stark was the daughter of a Lord\", we just obtain documents from a retriever." ] }, { @@ -587,14 +587,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Other use cases for Query Classifiers: custom classification models and zero shot classification\n", + "### Other use cases for Query Classifiers: custom classification models and zero-shot classification.\n", "\n", - "`TransformersQueryClassifier` is very flexible and also supports other possibilities for classifying queries.\n", - "For example, we may be interested in detecting the sentiment or classifying the topics. This can be done by loading a custom classification model from Transformers Hub or by using zero shot classification.\n", + "`TransformersQueryClassifier` is very flexible and also supports other options for classifying queries.\n", + "For example, we may be interested in detecting the sentiment or classifying the topics. We can do this by loading a custom classification model from the Transformers Hub or by using zero-shot classification.\n", "\n", - "#### Custom classification model vs zero shot classification\n", - "- traditional text classification models are trained to predict one of a few \"hard-coded\" classes and required a dedicated training dataset. In Transformers Hub you can find many pre-trained models, maybe even related to your domain of interest.\n", - "- zero-shot classification is very versatile: choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories." + "#### Custom classification model vs zero-shot classification\n", + "- traditional text classification models are trained to predict one of a few \"hard-coded\" classes and require a dedicated training dataset. In Transformers Hub, you can find many pre-trained models, maybe even related to your domain of interest.\n", + "- zero-shot classification is very versatile: by choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories." ] }, { @@ -602,10 +602,10 @@ "metadata": {}, "source": [ "#### Using custom classification models\n", - "We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose a appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.\n", + "We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.\n", "\n", "*In this case, the `labels` parameter must contain a list with the exact model labels.\n", - "The first provided label will corresponds to output_1, the second label to output_2, and so on.*" + "The first label we provide corresponds to output_1, the second label to output_2, and so on.*" ] }, { @@ -616,8 +616,8 @@ "source": [ "from haystack.nodes import TransformersQueryClassifier\n", "\n", - "# remember to compile a list with the exact model labels\n", - "# the first provided label will corresponds to output_1, the second label to output_2, and so on.\n", + "# Remember to compile a list with the exact model labels\n", + "# The first label you provide corresponds to output_1, the second label to output_2, and so on.\n", "labels = [\"LABEL_0\", \"LABEL_1\", \"LABEL_2\"]\n", "\n", "sentiment_query_classifier = TransformersQueryClassifier(\n", @@ -670,7 +670,7 @@ "metadata": {}, "source": [ "#### Using zero shot classification\n", - "It is also possible to perform zero shot classifcation, by providing a suitable base transformer model and **choosing** the classes that the model should predict.\n", + "You can also perform zero-shot classification by providing a suitable base transformer model and **choosing** the classes the model should predict.\n", "For example, we may be interested in whether the user query is related to music or cinema.\n", "\n", "*In this case, the `labels` parameter is a list containing the candidate classes.*" @@ -682,7 +682,7 @@ "metadata": {}, "outputs": [], "source": [ - "# in zero-shot-classification, the labels can be freely chosen\n", + "# In zero-shot-classification, you can choose the labels\n", "labels = [\"music\", \"cinema\"]\n", "\n", "query_classifier = TransformersQueryClassifier(\n", From d9e8d90c771318e0bcc80985fdbf1fd33c65d8ac Mon Sep 17 00:00:00 2001 From: anakin87 <44616784+anakin87@users.noreply.github.com> Date: Wed, 10 Aug 2022 11:46:16 +0200 Subject: [PATCH 05/18] add markdown --- docs/_src/tutorials/tutorials/14.md | 36 ++++++++++++++--------------- 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/_src/tutorials/tutorials/14.md b/docs/_src/tutorials/tutorials/14.md index a8ec89f16e..57d2af018b 100644 --- a/docs/_src/tutorials/tutorials/14.md +++ b/docs/_src/tutorials/tutorials/14.md @@ -83,7 +83,7 @@ from haystack.nodes import SklearnQueryClassifier keyword_classifier = SklearnQueryClassifier() ``` -Now let's feed some queries into this query classifier. We'll test with one keyword query, one interrogative query, and one statement query. Notice that we don't use any punctuation, such as question marks; this illustrates that the classifier doesn't need punctuation in order to make the right decision. +Now let's feed some queries into this query classifier. We'll test with one keyword query, one interrogative query, and one statement query. Note that we don't need to use any punctuation, such as question marks, for the query classifier to make the right decision. ```python @@ -94,7 +94,7 @@ queries = [ ] ``` -We can see below what our classifier does with these queries: "Arya Stark father" is rightly determined to be a keyword query and is sent to branch 2, while both the interrogative query "Who was the father of Arya Stark" and the statement query "Lord Eddard was the father of Arya Stark" are correctly labeled as non-keyword queries, and are thus shipped off to branch 1. +Below, you can see what the classifier does with these queries: it correctly determines that "Arya Stark father" is a keyword query and sends it to branch 2. It also correctly classifies both the interrogative query "Who was the father of Arya Stark" and the statement query "Lord Eddard was the father of Arya Stark" as non-keyword queries, and sends them to branch 1. ```python @@ -111,7 +111,7 @@ for query in queries: pd.DataFrame.from_dict(k_vs_qs_results) ``` -Next we will illustrate a **question vs. statement** `SklearnQueryClassifier`. We define our classifier below; notice that this time we have to explicitly specify the model and vectorizer, since the default for an `SklearnQueryClassifier` (and a `TransformersQueryClassifier`) is keyword vs. question/statement classification. +Next, we will illustrate a **question vs. statement** `SklearnQueryClassifier`. We define our classifier below. Note that this time we have to explicitly specify the model and vectorizer since the default for a `SklearnQueryClassifier` (and a `TransformersQueryClassifier`) is keyword vs. question/statement classification. ```python @@ -146,7 +146,7 @@ for query in queries: pd.DataFrame.from_dict(q_vs_s_results) ``` -And as we see, the question "Who was the father of Arya Stark" is sent to branch 1, while the statement "Lord Eddard was the father of Arya Stark" is sent to branch 2, so we can have our pipeline treat statements and questions differently. +And as we see, the question "Who was the father of Arya Stark" is sent to branch 1, while the statement "Lord Eddard was the father of Arya Stark" is sent to branch 2. This means we can have our pipeline treat statements and questions differently. ### Using Query Classifiers in a Pipeline @@ -237,7 +237,7 @@ sklearn_keyword_classifier.add_node(component=reader, name="QAReader", inputs=[" sklearn_keyword_classifier.draw("sklearn_keyword_classifier.png") ``` -Below we can see some results from this choice in branching structure: the keyword query "arya stark father" and the question query "Who is the father of Arya Stark?" generate noticeably different results, a distinction that is likely due to the use of different retrievers for keyword vs. question/statement queries. +Below, we can see how this choice affects the branching structure: the keyword query "arya stark father" and the question query "Who is the father of Arya Stark?" generate noticeably different results, a distinction that is likely due to the use of different retrievers for keyword vs. question/statement queries. ```python @@ -320,7 +320,7 @@ transformer_question_classifier.add_node(component=reader, name="QAReader", inpu transformer_question_classifier.draw("transformer_question_classifier.png") ``` -And below we see the results of this pipeline: with a question query like "Who is the father of Arya Stark?" we get back answers returned by a reader, but with a statement query like "Arya Stark was the daughter of a Lord" we just get back documents returned by a retriever. +And here are the results of this pipeline: with a question query like "Who is the father of Arya Stark?", we obtain answers from a reader, and with a statement query like "Arya Stark was the daughter of a Lord", we just obtain documents from a retriever. ```python @@ -339,27 +339,27 @@ print(f"\n\n{equal_line}\nSTATEMENT QUERY RESULTS\n{equal_line}") print_documents(res_2) ``` -### Other use cases for Query Classifiers: custom classification models and zero shot classification +### Other use cases for Query Classifiers: custom classification models and zero-shot classification. -`TransformersQueryClassifier` is very flexible and also supports other possibilities for classifying queries. -For example, we may be interested in detecting the sentiment or classifying the topics. This can be done by loading a custom classification model from Transformers Hub or by using zero shot classification. +`TransformersQueryClassifier` is very flexible and also supports other options for classifying queries. +For example, we may be interested in detecting the sentiment or classifying the topics. We can do this by loading a custom classification model from the Transformers Hub or by using zero-shot classification. -#### Custom classification model vs zero shot classification -- traditional text classification models are trained to predict one of a few "hard-coded" classes and required a dedicated training dataset. In Transformers Hub you can find many pre-trained models, maybe even related to your domain of interest. -- zero-shot classification is very versatile: choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories. +#### Custom classification model vs zero-shot classification +- traditional text classification models are trained to predict one of a few "hard-coded" classes and require a dedicated training dataset. In Transformers Hub, you can find many pre-trained models, maybe even related to your domain of interest. +- zero-shot classification is very versatile: by choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories. #### Using custom classification models -We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose a appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment. +We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment. *In this case, the `labels` parameter must contain a list with the exact model labels. -The first provided label will corresponds to output_1, the second label to output_2, and so on.* +The first label we provide corresponds to output_1, the second label to output_2, and so on.* ```python from haystack.nodes import TransformersQueryClassifier -# remember to compile a list with the exact model labels -# the first provided label will corresponds to output_1, the second label to output_2, and so on. +# Remember to compile a list with the exact model labels +# The first label you provide corresponds to output_1, the second label to output_2, and so on. labels = ["LABEL_0", "LABEL_1", "LABEL_2"] sentiment_query_classifier = TransformersQueryClassifier( @@ -400,14 +400,14 @@ pd.DataFrame.from_dict(sent_results) ``` #### Using zero shot classification -It is also possible to perform zero shot classifcation, by providing a suitable base transformer model and **choosing** the classes that the model should predict. +You can also perform zero-shot classification by providing a suitable base transformer model and **choosing** the classes the model should predict. For example, we may be interested in whether the user query is related to music or cinema. *In this case, the `labels` parameter is a list containing the candidate classes.* ```python -# in zero-shot-classification, the labels can be freely chosen +# In zero-shot-classification, you can choose the labels labels = ["music", "cinema"] query_classifier = TransformersQueryClassifier( From 5bec0a89594aa0be96861ac5591ccdbd1c6f1655 Mon Sep 17 00:00:00 2001 From: anakin87 <44616784+anakin87@users.noreply.github.com> Date: Tue, 9 Aug 2022 17:50:09 +0200 Subject: [PATCH 06/18] first draft for tutorial extension --- tutorials/Tutorial14_Query_Classifier.ipynb | 38 ++++++++------------- 1 file changed, 14 insertions(+), 24 deletions(-) diff --git a/tutorials/Tutorial14_Query_Classifier.ipynb b/tutorials/Tutorial14_Query_Classifier.ipynb index 592b96751e..de075dfa50 100644 --- a/tutorials/Tutorial14_Query_Classifier.ipynb +++ b/tutorials/Tutorial14_Query_Classifier.ipynb @@ -155,7 +155,7 @@ "id": "1NHjy9aa9FKx" }, "source": [ - "Now let's feed some queries into this query classifier. We'll test with one keyword query, one interrogative query, and one statement query. Note that we don't need to use any punctuation, such as question marks, for the query classifier to make the right decision." + "Now let's feed some queries into this query classifier. We'll test with one keyword query, one interrogative query, and one statement query. Notice that we don't use any punctuation, such as question marks; this illustrates that the classifier doesn't need punctuation in order to make the right decision." ] }, { @@ -179,7 +179,7 @@ "id": "UbKlyXcNj-nx" }, "source": [ - "Below, you can see what the classifier does with these queries: it correctly determines that \"Arya Stark father\" is a keyword query and sends it to branch 2. It also correctly classifies both the interrogative query \"Who was the father of Arya Stark\" and the statement query \"Lord Eddard was the father of Arya Stark\" as non-keyword queries, and sends them to branch 1." + "We can see below what our classifier does with these queries: \"Arya Stark father\" is rightly determined to be a keyword query and is sent to branch 2, while both the interrogative query \"Who was the father of Arya Stark\" and the statement query \"Lord Eddard was the father of Arya Stark\" are correctly labeled as non-keyword queries, and are thus shipped off to branch 1." ] }, { @@ -209,7 +209,7 @@ "id": "VyMZzRVHlG5O" }, "source": [ - "Next, we will illustrate a **question vs. statement** `SklearnQueryClassifier`. We define our classifier below. Note that this time we have to explicitly specify the model and vectorizer since the default for a `SklearnQueryClassifier` (and a `TransformersQueryClassifier`) is keyword vs. question/statement classification." + "Next we will illustrate a **question vs. statement** `SklearnQueryClassifier`. We define our classifier below; notice that this time we have to explicitly specify the model and vectorizer, since the default for an `SklearnQueryClassifier` (and a `TransformersQueryClassifier`) is keyword vs. question/statement classification." ] }, { @@ -271,7 +271,7 @@ "id": "Fk2kpvQR6Fa0" }, "source": [ - "And as we see, the question \"Who was the father of Arya Stark\" is sent to branch 1, while the statement \"Lord Eddard was the father of Arya Stark\" is sent to branch 2. This means we can have our pipeline treat statements and questions differently." + "And as we see, the question \"Who was the father of Arya Stark\" is sent to branch 1, while the statement \"Lord Eddard was the father of Arya Stark\" is sent to branch 2, so we can have our pipeline treat statements and questions differently." ] }, { @@ -433,7 +433,7 @@ "id": "imqRRCGTwQav" }, "source": [ - "Below, we can see how this choice affects the branching structure: the keyword query \"arya stark father\" and the question query \"Who is the father of Arya Stark?\" generate noticeably different results, a distinction that is likely due to the use of different retrievers for keyword vs. question/statement queries." + "Below we can see some results from this choice in branching structure: the keyword query \"arya stark father\" and the question query \"Who is the father of Arya Stark?\" generate noticeably different results, a distinction that is likely due to the use of different retrievers for keyword vs. question/statement queries." ] }, { @@ -557,7 +557,7 @@ "id": "QU1B6JQEDrol" }, "source": [ - "And here are the results of this pipeline: with a question query like \"Who is the father of Arya Stark?\", we obtain answers from a reader, and with a statement query like \"Arya Stark was the daughter of a Lord\", we just obtain documents from a retriever." + "And below we see the results of this pipeline: with a question query like \"Who is the father of Arya Stark?\" we get back answers returned by a reader, but with a statement query like \"Arya Stark was the daughter of a Lord\" we just get back documents returned by a retriever." ] }, { @@ -587,14 +587,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Other use cases for Query Classifiers: custom classification models and zero-shot classification.\n", + "### Other use cases for Query Classifiers: custom classification models and zero shot classification\n", "\n", - "`TransformersQueryClassifier` is very flexible and also supports other options for classifying queries.\n", - "For example, we may be interested in detecting the sentiment or classifying the topics. We can do this by loading a custom classification model from the Transformers Hub or by using zero-shot classification.\n", - "\n", - "#### Custom classification model vs zero-shot classification\n", - "- traditional text classification models are trained to predict one of a few \"hard-coded\" classes and require a dedicated training dataset. In Transformers Hub, you can find many pre-trained models, maybe even related to your domain of interest.\n", - "- zero-shot classification is very versatile: by choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories." + "`TransformersQueryClassifier` is very flexible and also supports other possibilities for classifying queries, including loading a custom classification model from Transformers Hub or using zero shot classification." ] }, { @@ -602,10 +597,7 @@ "metadata": {}, "source": [ "#### Using custom classification models\n", - "We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.\n", - "\n", - "*In this case, the `labels` parameter must contain a list with the exact model labels.\n", - "The first label we provide corresponds to output_1, the second label to output_2, and so on.*" + "We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose a appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.### Using zero-shot-classification" ] }, { @@ -616,8 +608,8 @@ "source": [ "from haystack.nodes import TransformersQueryClassifier\n", "\n", - "# Remember to compile a list with the exact model labels\n", - "# The first label you provide corresponds to output_1, the second label to output_2, and so on.\n", + "# remember to compile a list with the exact model labels\n", + "# the first provided label will corresponds to output_1, the second label to output_2, and so on.\n", "labels = [\"LABEL_0\", \"LABEL_1\", \"LABEL_2\"]\n", "\n", "sentiment_query_classifier = TransformersQueryClassifier(\n", @@ -670,10 +662,8 @@ "metadata": {}, "source": [ "#### Using zero shot classification\n", - "You can also perform zero-shot classification by providing a suitable base transformer model and **choosing** the classes the model should predict.\n", - "For example, we may be interested in whether the user query is related to music or cinema.\n", - "\n", - "*In this case, the `labels` parameter is a list containing the candidate classes.*" + "It is also possible to perform zero shot classifcation, by providing a suitable base transformer model and **choosing** the classes that the model should predict.\n", + "For example, we may be interested in whether the user query is related to music or cinema." ] }, { @@ -682,7 +672,7 @@ "metadata": {}, "outputs": [], "source": [ - "# In zero-shot-classification, you can choose the labels\n", + "# in zero-shot-classification, the labels can be freely chosen\n", "labels = [\"music\", \"cinema\"]\n", "\n", "query_classifier = TransformersQueryClassifier(\n", From aa290a3f2f710e59b2c74885d4edb135f67789a3 Mon Sep 17 00:00:00 2001 From: anakin87 <44616784+anakin87@users.noreply.github.com> Date: Tue, 9 Aug 2022 18:06:15 +0200 Subject: [PATCH 07/18] forgotten markdown --- docs/_src/tutorials/tutorials/14.md | 24 +++++++----------------- 1 file changed, 7 insertions(+), 17 deletions(-) diff --git a/docs/_src/tutorials/tutorials/14.md b/docs/_src/tutorials/tutorials/14.md index 57d2af018b..871090b011 100644 --- a/docs/_src/tutorials/tutorials/14.md +++ b/docs/_src/tutorials/tutorials/14.md @@ -339,27 +339,19 @@ print(f"\n\n{equal_line}\nSTATEMENT QUERY RESULTS\n{equal_line}") print_documents(res_2) ``` -### Other use cases for Query Classifiers: custom classification models and zero-shot classification. +### Other use cases for Query Classifiers: custom classification models and zero shot classification -`TransformersQueryClassifier` is very flexible and also supports other options for classifying queries. -For example, we may be interested in detecting the sentiment or classifying the topics. We can do this by loading a custom classification model from the Transformers Hub or by using zero-shot classification. - -#### Custom classification model vs zero-shot classification -- traditional text classification models are trained to predict one of a few "hard-coded" classes and require a dedicated training dataset. In Transformers Hub, you can find many pre-trained models, maybe even related to your domain of interest. -- zero-shot classification is very versatile: by choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories. +`TransformersQueryClassifier` is very flexible and also supports other possibilities for classifying queries, including loading a custom classification model from Transformers Hub or using zero shot classification. #### Using custom classification models -We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment. - -*In this case, the `labels` parameter must contain a list with the exact model labels. -The first label we provide corresponds to output_1, the second label to output_2, and so on.* +We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose a appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.### Using zero-shot-classification ```python from haystack.nodes import TransformersQueryClassifier -# Remember to compile a list with the exact model labels -# The first label you provide corresponds to output_1, the second label to output_2, and so on. +# remember to compile a list with the exact model labels +# the first provided label will corresponds to output_1, the second label to output_2, and so on. labels = ["LABEL_0", "LABEL_1", "LABEL_2"] sentiment_query_classifier = TransformersQueryClassifier( @@ -400,14 +392,12 @@ pd.DataFrame.from_dict(sent_results) ``` #### Using zero shot classification -You can also perform zero-shot classification by providing a suitable base transformer model and **choosing** the classes the model should predict. +It is also possible to perform zero shot classifcation, by providing a suitable base transformer model and **choosing** the classes that the model should predict. For example, we may be interested in whether the user query is related to music or cinema. -*In this case, the `labels` parameter is a list containing the candidate classes.* - ```python -# In zero-shot-classification, you can choose the labels +# in zero-shot-classification, the labels can be freely chosen labels = ["music", "cinema"] query_classifier = TransformersQueryClassifier( From 44e0fdae7e35b45a4545927bc4b45a9ec4520920 Mon Sep 17 00:00:00 2001 From: anakin87 <44616784+anakin87@users.noreply.github.com> Date: Wed, 10 Aug 2022 09:53:57 +0200 Subject: [PATCH 08/18] improved tutorial --- docs/_src/tutorials/tutorials/14.md | 14 ++++++++++++-- tutorials/Tutorial14_Query_Classifier.ipynb | 16 +++++++++++++--- 2 files changed, 25 insertions(+), 5 deletions(-) diff --git a/docs/_src/tutorials/tutorials/14.md b/docs/_src/tutorials/tutorials/14.md index 871090b011..bab397445d 100644 --- a/docs/_src/tutorials/tutorials/14.md +++ b/docs/_src/tutorials/tutorials/14.md @@ -341,10 +341,18 @@ print_documents(res_2) ### Other use cases for Query Classifiers: custom classification models and zero shot classification -`TransformersQueryClassifier` is very flexible and also supports other possibilities for classifying queries, including loading a custom classification model from Transformers Hub or using zero shot classification. +`TransformersQueryClassifier` is very flexible and also supports other possibilities for classifying queries. +For example, we may be interested in detecting the sentiment or classifying the topics. This can be done by loading a custom classification model from Transformers Hub or by using zero shot classification. + +#### Custom classification model vs zero shot classification +- traditional text classification models are trained to predict one of a few "hard-coded" classes and required a dedicated training dataset. In Transformers Hub you can find many pre-trained models, maybe even related to your domain of interest. +- zero-shot classification is very versatile: choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories. #### Using custom classification models -We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose a appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.### Using zero-shot-classification +We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose a appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment. + +*In this case, the `labels` parameter must contain a list with the exact model labels. +The first provided label will corresponds to output_1, the second label to output_2, and so on.* ```python @@ -395,6 +403,8 @@ pd.DataFrame.from_dict(sent_results) It is also possible to perform zero shot classifcation, by providing a suitable base transformer model and **choosing** the classes that the model should predict. For example, we may be interested in whether the user query is related to music or cinema. +*In this case, the `labels` parameter is a list containing the candidate classes.* + ```python # in zero-shot-classification, the labels can be freely chosen diff --git a/tutorials/Tutorial14_Query_Classifier.ipynb b/tutorials/Tutorial14_Query_Classifier.ipynb index de075dfa50..0290c1fa0b 100644 --- a/tutorials/Tutorial14_Query_Classifier.ipynb +++ b/tutorials/Tutorial14_Query_Classifier.ipynb @@ -589,7 +589,12 @@ "source": [ "### Other use cases for Query Classifiers: custom classification models and zero shot classification\n", "\n", - "`TransformersQueryClassifier` is very flexible and also supports other possibilities for classifying queries, including loading a custom classification model from Transformers Hub or using zero shot classification." + "`TransformersQueryClassifier` is very flexible and also supports other possibilities for classifying queries.\n", + "For example, we may be interested in detecting the sentiment or classifying the topics. This can be done by loading a custom classification model from Transformers Hub or by using zero shot classification.\n", + "\n", + "#### Custom classification model vs zero shot classification\n", + "- traditional text classification models are trained to predict one of a few \"hard-coded\" classes and required a dedicated training dataset. In Transformers Hub you can find many pre-trained models, maybe even related to your domain of interest.\n", + "- zero-shot classification is very versatile: choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories." ] }, { @@ -597,7 +602,10 @@ "metadata": {}, "source": [ "#### Using custom classification models\n", - "We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose a appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.### Using zero-shot-classification" + "We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose a appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.\n", + "\n", + "*In this case, the `labels` parameter must contain a list with the exact model labels.\n", + "The first provided label will corresponds to output_1, the second label to output_2, and so on.*" ] }, { @@ -663,7 +671,9 @@ "source": [ "#### Using zero shot classification\n", "It is also possible to perform zero shot classifcation, by providing a suitable base transformer model and **choosing** the classes that the model should predict.\n", - "For example, we may be interested in whether the user query is related to music or cinema." + "For example, we may be interested in whether the user query is related to music or cinema.\n", + "\n", + "*In this case, the `labels` parameter is a list containing the candidate classes.*" ] }, { From e13f247703ed778b1ebb4936978e58be2ca857c4 Mon Sep 17 00:00:00 2001 From: Stefano Fiorucci <44616784+anakin87@users.noreply.github.com> Date: Wed, 10 Aug 2022 11:43:26 +0200 Subject: [PATCH 09/18] Apply suggestions from code review Co-authored-by: Agnieszka Marzec <97166305+agnieszka-m@users.noreply.github.com> --- tutorials/Tutorial14_Query_Classifier.ipynb | 36 ++++++++++----------- 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/tutorials/Tutorial14_Query_Classifier.ipynb b/tutorials/Tutorial14_Query_Classifier.ipynb index 0290c1fa0b..592b96751e 100644 --- a/tutorials/Tutorial14_Query_Classifier.ipynb +++ b/tutorials/Tutorial14_Query_Classifier.ipynb @@ -155,7 +155,7 @@ "id": "1NHjy9aa9FKx" }, "source": [ - "Now let's feed some queries into this query classifier. We'll test with one keyword query, one interrogative query, and one statement query. Notice that we don't use any punctuation, such as question marks; this illustrates that the classifier doesn't need punctuation in order to make the right decision." + "Now let's feed some queries into this query classifier. We'll test with one keyword query, one interrogative query, and one statement query. Note that we don't need to use any punctuation, such as question marks, for the query classifier to make the right decision." ] }, { @@ -179,7 +179,7 @@ "id": "UbKlyXcNj-nx" }, "source": [ - "We can see below what our classifier does with these queries: \"Arya Stark father\" is rightly determined to be a keyword query and is sent to branch 2, while both the interrogative query \"Who was the father of Arya Stark\" and the statement query \"Lord Eddard was the father of Arya Stark\" are correctly labeled as non-keyword queries, and are thus shipped off to branch 1." + "Below, you can see what the classifier does with these queries: it correctly determines that \"Arya Stark father\" is a keyword query and sends it to branch 2. It also correctly classifies both the interrogative query \"Who was the father of Arya Stark\" and the statement query \"Lord Eddard was the father of Arya Stark\" as non-keyword queries, and sends them to branch 1." ] }, { @@ -209,7 +209,7 @@ "id": "VyMZzRVHlG5O" }, "source": [ - "Next we will illustrate a **question vs. statement** `SklearnQueryClassifier`. We define our classifier below; notice that this time we have to explicitly specify the model and vectorizer, since the default for an `SklearnQueryClassifier` (and a `TransformersQueryClassifier`) is keyword vs. question/statement classification." + "Next, we will illustrate a **question vs. statement** `SklearnQueryClassifier`. We define our classifier below. Note that this time we have to explicitly specify the model and vectorizer since the default for a `SklearnQueryClassifier` (and a `TransformersQueryClassifier`) is keyword vs. question/statement classification." ] }, { @@ -271,7 +271,7 @@ "id": "Fk2kpvQR6Fa0" }, "source": [ - "And as we see, the question \"Who was the father of Arya Stark\" is sent to branch 1, while the statement \"Lord Eddard was the father of Arya Stark\" is sent to branch 2, so we can have our pipeline treat statements and questions differently." + "And as we see, the question \"Who was the father of Arya Stark\" is sent to branch 1, while the statement \"Lord Eddard was the father of Arya Stark\" is sent to branch 2. This means we can have our pipeline treat statements and questions differently." ] }, { @@ -433,7 +433,7 @@ "id": "imqRRCGTwQav" }, "source": [ - "Below we can see some results from this choice in branching structure: the keyword query \"arya stark father\" and the question query \"Who is the father of Arya Stark?\" generate noticeably different results, a distinction that is likely due to the use of different retrievers for keyword vs. question/statement queries." + "Below, we can see how this choice affects the branching structure: the keyword query \"arya stark father\" and the question query \"Who is the father of Arya Stark?\" generate noticeably different results, a distinction that is likely due to the use of different retrievers for keyword vs. question/statement queries." ] }, { @@ -557,7 +557,7 @@ "id": "QU1B6JQEDrol" }, "source": [ - "And below we see the results of this pipeline: with a question query like \"Who is the father of Arya Stark?\" we get back answers returned by a reader, but with a statement query like \"Arya Stark was the daughter of a Lord\" we just get back documents returned by a retriever." + "And here are the results of this pipeline: with a question query like \"Who is the father of Arya Stark?\", we obtain answers from a reader, and with a statement query like \"Arya Stark was the daughter of a Lord\", we just obtain documents from a retriever." ] }, { @@ -587,14 +587,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Other use cases for Query Classifiers: custom classification models and zero shot classification\n", + "### Other use cases for Query Classifiers: custom classification models and zero-shot classification.\n", "\n", - "`TransformersQueryClassifier` is very flexible and also supports other possibilities for classifying queries.\n", - "For example, we may be interested in detecting the sentiment or classifying the topics. This can be done by loading a custom classification model from Transformers Hub or by using zero shot classification.\n", + "`TransformersQueryClassifier` is very flexible and also supports other options for classifying queries.\n", + "For example, we may be interested in detecting the sentiment or classifying the topics. We can do this by loading a custom classification model from the Transformers Hub or by using zero-shot classification.\n", "\n", - "#### Custom classification model vs zero shot classification\n", - "- traditional text classification models are trained to predict one of a few \"hard-coded\" classes and required a dedicated training dataset. In Transformers Hub you can find many pre-trained models, maybe even related to your domain of interest.\n", - "- zero-shot classification is very versatile: choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories." + "#### Custom classification model vs zero-shot classification\n", + "- traditional text classification models are trained to predict one of a few \"hard-coded\" classes and require a dedicated training dataset. In Transformers Hub, you can find many pre-trained models, maybe even related to your domain of interest.\n", + "- zero-shot classification is very versatile: by choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories." ] }, { @@ -602,10 +602,10 @@ "metadata": {}, "source": [ "#### Using custom classification models\n", - "We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose a appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.\n", + "We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.\n", "\n", "*In this case, the `labels` parameter must contain a list with the exact model labels.\n", - "The first provided label will corresponds to output_1, the second label to output_2, and so on.*" + "The first label we provide corresponds to output_1, the second label to output_2, and so on.*" ] }, { @@ -616,8 +616,8 @@ "source": [ "from haystack.nodes import TransformersQueryClassifier\n", "\n", - "# remember to compile a list with the exact model labels\n", - "# the first provided label will corresponds to output_1, the second label to output_2, and so on.\n", + "# Remember to compile a list with the exact model labels\n", + "# The first label you provide corresponds to output_1, the second label to output_2, and so on.\n", "labels = [\"LABEL_0\", \"LABEL_1\", \"LABEL_2\"]\n", "\n", "sentiment_query_classifier = TransformersQueryClassifier(\n", @@ -670,7 +670,7 @@ "metadata": {}, "source": [ "#### Using zero shot classification\n", - "It is also possible to perform zero shot classifcation, by providing a suitable base transformer model and **choosing** the classes that the model should predict.\n", + "You can also perform zero-shot classification by providing a suitable base transformer model and **choosing** the classes the model should predict.\n", "For example, we may be interested in whether the user query is related to music or cinema.\n", "\n", "*In this case, the `labels` parameter is a list containing the candidate classes.*" @@ -682,7 +682,7 @@ "metadata": {}, "outputs": [], "source": [ - "# in zero-shot-classification, the labels can be freely chosen\n", + "# In zero-shot-classification, you can choose the labels\n", "labels = [\"music\", \"cinema\"]\n", "\n", "query_classifier = TransformersQueryClassifier(\n", From cd2413c3f508a0b3b65882e2ba9bda455d378fee Mon Sep 17 00:00:00 2001 From: anakin87 <44616784+anakin87@users.noreply.github.com> Date: Wed, 10 Aug 2022 11:46:16 +0200 Subject: [PATCH 10/18] add markdown --- docs/_src/tutorials/tutorials/14.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/_src/tutorials/tutorials/14.md b/docs/_src/tutorials/tutorials/14.md index bab397445d..57d2af018b 100644 --- a/docs/_src/tutorials/tutorials/14.md +++ b/docs/_src/tutorials/tutorials/14.md @@ -339,27 +339,27 @@ print(f"\n\n{equal_line}\nSTATEMENT QUERY RESULTS\n{equal_line}") print_documents(res_2) ``` -### Other use cases for Query Classifiers: custom classification models and zero shot classification +### Other use cases for Query Classifiers: custom classification models and zero-shot classification. -`TransformersQueryClassifier` is very flexible and also supports other possibilities for classifying queries. -For example, we may be interested in detecting the sentiment or classifying the topics. This can be done by loading a custom classification model from Transformers Hub or by using zero shot classification. +`TransformersQueryClassifier` is very flexible and also supports other options for classifying queries. +For example, we may be interested in detecting the sentiment or classifying the topics. We can do this by loading a custom classification model from the Transformers Hub or by using zero-shot classification. -#### Custom classification model vs zero shot classification -- traditional text classification models are trained to predict one of a few "hard-coded" classes and required a dedicated training dataset. In Transformers Hub you can find many pre-trained models, maybe even related to your domain of interest. -- zero-shot classification is very versatile: choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories. +#### Custom classification model vs zero-shot classification +- traditional text classification models are trained to predict one of a few "hard-coded" classes and require a dedicated training dataset. In Transformers Hub, you can find many pre-trained models, maybe even related to your domain of interest. +- zero-shot classification is very versatile: by choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories. #### Using custom classification models -We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose a appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment. +We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment. *In this case, the `labels` parameter must contain a list with the exact model labels. -The first provided label will corresponds to output_1, the second label to output_2, and so on.* +The first label we provide corresponds to output_1, the second label to output_2, and so on.* ```python from haystack.nodes import TransformersQueryClassifier -# remember to compile a list with the exact model labels -# the first provided label will corresponds to output_1, the second label to output_2, and so on. +# Remember to compile a list with the exact model labels +# The first label you provide corresponds to output_1, the second label to output_2, and so on. labels = ["LABEL_0", "LABEL_1", "LABEL_2"] sentiment_query_classifier = TransformersQueryClassifier( @@ -400,14 +400,14 @@ pd.DataFrame.from_dict(sent_results) ``` #### Using zero shot classification -It is also possible to perform zero shot classifcation, by providing a suitable base transformer model and **choosing** the classes that the model should predict. +You can also perform zero-shot classification by providing a suitable base transformer model and **choosing** the classes the model should predict. For example, we may be interested in whether the user query is related to music or cinema. *In this case, the `labels` parameter is a list containing the candidate classes.* ```python -# in zero-shot-classification, the labels can be freely chosen +# In zero-shot-classification, you can choose the labels labels = ["music", "cinema"] query_classifier = TransformersQueryClassifier( From 1a99cbd050e2c29c192d79080dd168be1f432e4a Mon Sep 17 00:00:00 2001 From: anakin87 <44616784+anakin87@users.noreply.github.com> Date: Wed, 10 Aug 2022 22:00:44 +0200 Subject: [PATCH 11/18] little corrections --- tutorials/Tutorial14_Query_Classifier.ipynb | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tutorials/Tutorial14_Query_Classifier.ipynb b/tutorials/Tutorial14_Query_Classifier.ipynb index 592b96751e..303f423c42 100644 --- a/tutorials/Tutorial14_Query_Classifier.ipynb +++ b/tutorials/Tutorial14_Query_Classifier.ipynb @@ -590,10 +590,10 @@ "### Other use cases for Query Classifiers: custom classification models and zero-shot classification.\n", "\n", "`TransformersQueryClassifier` is very flexible and also supports other options for classifying queries.\n", - "For example, we may be interested in detecting the sentiment or classifying the topics. We can do this by loading a custom classification model from the Transformers Hub or by using zero-shot classification.\n", + "For example, we may be interested in detecting the sentiment or classifying the topics. We can do this by loading a custom classification model from the Huggingface Hub or by using zero-shot classification.\n", "\n", "#### Custom classification model vs zero-shot classification\n", - "- traditional text classification models are trained to predict one of a few \"hard-coded\" classes and require a dedicated training dataset. In Transformers Hub, you can find many pre-trained models, maybe even related to your domain of interest.\n", + "- traditional text classification models are trained to predict one of a few \"hard-coded\" classes and require a dedicated training dataset. In Huggingface Hub, you can find many pre-trained models, maybe even related to your domain of interest.\n", "- zero-shot classification is very versatile: by choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories." ] }, @@ -602,7 +602,7 @@ "metadata": {}, "source": [ "#### Using custom classification models\n", - "We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.\n", + "We can use a public model, available in the Huggingface Hub. For example, we might be interested in classifying the sentiment of the queries, so we choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.\n", "\n", "*In this case, the `labels` parameter must contain a list with the exact model labels.\n", "The first label we provide corresponds to output_1, the second label to output_2, and so on.*" From e1bf48613c9fd0873660e38eb54c01b73bb7a402 Mon Sep 17 00:00:00 2001 From: anakin87 <44616784+anakin87@users.noreply.github.com> Date: Wed, 10 Aug 2022 22:46:32 +0200 Subject: [PATCH 12/18] little corrections and add py tutorial --- docs/_src/tutorials/tutorials/14.md | 8 +-- tutorials/Tutorial14_Query_Classifier.ipynb | 2 +- tutorials/Tutorial14_Query_Classifier.py | 68 +++++++++++++++++++++ 3 files changed, 73 insertions(+), 5 deletions(-) diff --git a/docs/_src/tutorials/tutorials/14.md b/docs/_src/tutorials/tutorials/14.md index 57d2af018b..5e584bbc5d 100644 --- a/docs/_src/tutorials/tutorials/14.md +++ b/docs/_src/tutorials/tutorials/14.md @@ -342,14 +342,14 @@ print_documents(res_2) ### Other use cases for Query Classifiers: custom classification models and zero-shot classification. `TransformersQueryClassifier` is very flexible and also supports other options for classifying queries. -For example, we may be interested in detecting the sentiment or classifying the topics. We can do this by loading a custom classification model from the Transformers Hub or by using zero-shot classification. +For example, we may be interested in detecting the sentiment or classifying the topics. We can do this by loading a custom classification model from the Huggingface Hub or by using zero-shot classification. #### Custom classification model vs zero-shot classification -- traditional text classification models are trained to predict one of a few "hard-coded" classes and require a dedicated training dataset. In Transformers Hub, you can find many pre-trained models, maybe even related to your domain of interest. +- traditional text classification models are trained to predict one of a few "hard-coded" classes and require a dedicated training dataset. In Huggingface Hub, you can find many pre-trained models, maybe even related to your domain of interest. - zero-shot classification is very versatile: by choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories. #### Using custom classification models -We can use a public model, available in the Transformer Hub. For example, we might be interested in classifying the sentiment of the *queries*, so we choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment. +We can use a public model, available in the Huggingface Hub. For example, we might be interested in classifying the sentiment of the queries, so we choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment. *In this case, the `labels` parameter must contain a list with the exact model labels. The first label we provide corresponds to output_1, the second label to output_2, and so on.* @@ -399,7 +399,7 @@ for query in queries: pd.DataFrame.from_dict(sent_results) ``` -#### Using zero shot classification +#### Using zero-shot classification You can also perform zero-shot classification by providing a suitable base transformer model and **choosing** the classes the model should predict. For example, we may be interested in whether the user query is related to music or cinema. diff --git a/tutorials/Tutorial14_Query_Classifier.ipynb b/tutorials/Tutorial14_Query_Classifier.ipynb index 303f423c42..30bbdb3cfb 100644 --- a/tutorials/Tutorial14_Query_Classifier.ipynb +++ b/tutorials/Tutorial14_Query_Classifier.ipynb @@ -669,7 +669,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "#### Using zero shot classification\n", + "#### Using zero-shot classification\n", "You can also perform zero-shot classification by providing a suitable base transformer model and **choosing** the classes the model should predict.\n", "For example, we may be interested in whether the user query is related to music or cinema.\n", "\n", diff --git a/tutorials/Tutorial14_Query_Classifier.py b/tutorials/Tutorial14_Query_Classifier.py index 30f360a18b..3863eaa8b0 100644 --- a/tutorials/Tutorial14_Query_Classifier.py +++ b/tutorials/Tutorial14_Query_Classifier.py @@ -179,6 +179,74 @@ def print_header(header): print_documents(res_2) print("") + # Other use cases for Query Classifiers + + # Custom classification models + + # Remember to compile a list with the exact model labels + # The first label you provide corresponds to output_1, the second label to output_2, and so on. + labels = ["LABEL_0", "LABEL_1", "LABEL_2"] + + sentiment_query_classifier = TransformersQueryClassifier( + model_name_or_path="cardiffnlp/twitter-roberta-base-sentiment", + use_gpu=True, + task="text-classification", + labels=labels, + ) + + queries = [ + "What's the answer?", # neutral query + "Would you be so lovely to tell me the answer?", # positive query + "Can you give me the damn right answer for once??", # negative query + ] + + sent_results = {"Query": [], "Output Branch": [], "Class": []} + + for query in queries: + result = sentiment_query_classifier.run(query=query) + sent_results["Query"].append(query) + sent_results["Output Branch"].append(result[1]) + if result[1] == "output_1": + sent_results["Class"].append("negative") + elif result[1] == "output_2": + sent_results["Class"].append("neutral") + elif result[1] == "output_3": + sent_results["Class"].append("positive") + + print_header("Query Sentiment Classification with custom transformer model") + print(pd.DataFrame.from_dict(sent_results)) + print("") + + # Zero-shot classification + + # In zero-shot-classification, you can choose the labels + labels = ["music", "cinema"] + + query_classifier = TransformersQueryClassifier( + model_name_or_path="typeform/distilbert-base-uncased-mnli", + use_gpu=True, + task="zero-shot-classification", + labels=labels, + ) + + queries = [ + "In which films does John Travolta appear?", # query about cinema + "What is the Rolling Stones first album?", # query about music + "Who was Sergio Leone?", # query about cinema + ] + + query_classification_results = {"Query": [], "Output Branch": [], "Class": []} + + for query in queries: + result = query_classifier.run(query=query) + query_classification_results["Query"].append(query) + query_classification_results["Output Branch"].append(result[1]) + query_classification_results["Class"].append("music" if result[1] == "output_1" else "cinema") + + print_header("Query Zero-shot Classification") + print(pd.DataFrame.from_dict(query_classification_results)) + print("") + if __name__ == "__main__": tutorial14_query_classifier() From 2d25ced00694f2d9f58f3f984a087313a303f3cf Mon Sep 17 00:00:00 2001 From: Stefano Fiorucci <44616784+anakin87@users.noreply.github.com> Date: Thu, 11 Aug 2022 18:45:27 +0200 Subject: [PATCH 13/18] Update tutorials/Tutorial14_Query_Classifier.ipynb Co-authored-by: Agnieszka Marzec <97166305+agnieszka-m@users.noreply.github.com> --- tutorials/Tutorial14_Query_Classifier.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tutorials/Tutorial14_Query_Classifier.ipynb b/tutorials/Tutorial14_Query_Classifier.ipynb index 30bbdb3cfb..2c2f0de3f4 100644 --- a/tutorials/Tutorial14_Query_Classifier.ipynb +++ b/tutorials/Tutorial14_Query_Classifier.ipynb @@ -590,7 +590,7 @@ "### Other use cases for Query Classifiers: custom classification models and zero-shot classification.\n", "\n", "`TransformersQueryClassifier` is very flexible and also supports other options for classifying queries.\n", - "For example, we may be interested in detecting the sentiment or classifying the topics. We can do this by loading a custom classification model from the Huggingface Hub or by using zero-shot classification.\n", + "For example, we may be interested in detecting the sentiment or classifying the topics. We can do this by loading a custom classification model from the Hugging Face Hub or by using zero-shot classification.\n", "\n", "#### Custom classification model vs zero-shot classification\n", "- traditional text classification models are trained to predict one of a few \"hard-coded\" classes and require a dedicated training dataset. In Huggingface Hub, you can find many pre-trained models, maybe even related to your domain of interest.\n", From 0891fa55911e25775391a08c97184456f15aaed4 Mon Sep 17 00:00:00 2001 From: Stefano Fiorucci <44616784+anakin87@users.noreply.github.com> Date: Thu, 11 Aug 2022 18:45:41 +0200 Subject: [PATCH 14/18] Update tutorials/Tutorial14_Query_Classifier.ipynb Co-authored-by: Agnieszka Marzec <97166305+agnieszka-m@users.noreply.github.com> --- tutorials/Tutorial14_Query_Classifier.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tutorials/Tutorial14_Query_Classifier.ipynb b/tutorials/Tutorial14_Query_Classifier.ipynb index 2c2f0de3f4..e5ee65a98d 100644 --- a/tutorials/Tutorial14_Query_Classifier.ipynb +++ b/tutorials/Tutorial14_Query_Classifier.ipynb @@ -593,7 +593,7 @@ "For example, we may be interested in detecting the sentiment or classifying the topics. We can do this by loading a custom classification model from the Hugging Face Hub or by using zero-shot classification.\n", "\n", "#### Custom classification model vs zero-shot classification\n", - "- traditional text classification models are trained to predict one of a few \"hard-coded\" classes and require a dedicated training dataset. In Huggingface Hub, you can find many pre-trained models, maybe even related to your domain of interest.\n", + "- Rraditional text classification models are trained to predict one of a few \"hard-coded\" classes and require a dedicated training dataset. In the Hugging Face Hub, you can find many pre-trained models, maybe even related to your domain of interest.\n", "- zero-shot classification is very versatile: by choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories." ] }, From 09ca7df92f8ee3ff7ab6e426055efe4b9f693039 Mon Sep 17 00:00:00 2001 From: Stefano Fiorucci <44616784+anakin87@users.noreply.github.com> Date: Thu, 11 Aug 2022 18:45:48 +0200 Subject: [PATCH 15/18] Update tutorials/Tutorial14_Query_Classifier.ipynb Co-authored-by: Agnieszka Marzec <97166305+agnieszka-m@users.noreply.github.com> --- tutorials/Tutorial14_Query_Classifier.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tutorials/Tutorial14_Query_Classifier.ipynb b/tutorials/Tutorial14_Query_Classifier.ipynb index e5ee65a98d..d953c0dfa4 100644 --- a/tutorials/Tutorial14_Query_Classifier.ipynb +++ b/tutorials/Tutorial14_Query_Classifier.ipynb @@ -594,7 +594,7 @@ "\n", "#### Custom classification model vs zero-shot classification\n", "- Rraditional text classification models are trained to predict one of a few \"hard-coded\" classes and require a dedicated training dataset. In the Hugging Face Hub, you can find many pre-trained models, maybe even related to your domain of interest.\n", - "- zero-shot classification is very versatile: by choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories." + "- Zero-shot classification is very versatile: by choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories." ] }, { From c75ff894866ddbd59958fc0b7fe1311bb99114e8 Mon Sep 17 00:00:00 2001 From: Stefano Fiorucci <44616784+anakin87@users.noreply.github.com> Date: Thu, 11 Aug 2022 18:45:56 +0200 Subject: [PATCH 16/18] Update tutorials/Tutorial14_Query_Classifier.ipynb Co-authored-by: Agnieszka Marzec <97166305+agnieszka-m@users.noreply.github.com> --- tutorials/Tutorial14_Query_Classifier.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tutorials/Tutorial14_Query_Classifier.ipynb b/tutorials/Tutorial14_Query_Classifier.ipynb index d953c0dfa4..60ab0f7792 100644 --- a/tutorials/Tutorial14_Query_Classifier.ipynb +++ b/tutorials/Tutorial14_Query_Classifier.ipynb @@ -602,7 +602,7 @@ "metadata": {}, "source": [ "#### Using custom classification models\n", - "We can use a public model, available in the Huggingface Hub. For example, we might be interested in classifying the sentiment of the queries, so we choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.\n", + "We can use a public model, available in the Hugging Face Hub. For example, if we want to classify the sentiment of the queries, we cam choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.\n", "\n", "*In this case, the `labels` parameter must contain a list with the exact model labels.\n", "The first label we provide corresponds to output_1, the second label to output_2, and so on.*" From b7114b84bfc041d3a0769c8dba81ac5311e05349 Mon Sep 17 00:00:00 2001 From: Thomas Stadelmann Date: Thu, 11 Aug 2022 19:27:01 +0200 Subject: [PATCH 17/18] update tutorial webpage --- docs/_src/tutorials/tutorials/14.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/_src/tutorials/tutorials/14.md b/docs/_src/tutorials/tutorials/14.md index 5e584bbc5d..98fed2995c 100644 --- a/docs/_src/tutorials/tutorials/14.md +++ b/docs/_src/tutorials/tutorials/14.md @@ -342,14 +342,14 @@ print_documents(res_2) ### Other use cases for Query Classifiers: custom classification models and zero-shot classification. `TransformersQueryClassifier` is very flexible and also supports other options for classifying queries. -For example, we may be interested in detecting the sentiment or classifying the topics. We can do this by loading a custom classification model from the Huggingface Hub or by using zero-shot classification. +For example, we may be interested in detecting the sentiment or classifying the topics. We can do this by loading a custom classification model from the Hugging Face Hub or by using zero-shot classification. #### Custom classification model vs zero-shot classification -- traditional text classification models are trained to predict one of a few "hard-coded" classes and require a dedicated training dataset. In Huggingface Hub, you can find many pre-trained models, maybe even related to your domain of interest. -- zero-shot classification is very versatile: by choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories. +- Rraditional text classification models are trained to predict one of a few "hard-coded" classes and require a dedicated training dataset. In the Hugging Face Hub, you can find many pre-trained models, maybe even related to your domain of interest. +- Zero-shot classification is very versatile: by choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories. #### Using custom classification models -We can use a public model, available in the Huggingface Hub. For example, we might be interested in classifying the sentiment of the queries, so we choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment. +We can use a public model, available in the Hugging Face Hub. For example, if we want to classify the sentiment of the queries, we cam choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment. *In this case, the `labels` parameter must contain a list with the exact model labels. The first label we provide corresponds to output_1, the second label to output_2, and so on.* From 55bf7eedf08b0dd1450f28fb4d046513e80093bf Mon Sep 17 00:00:00 2001 From: Thomas Stadelmann Date: Fri, 12 Aug 2022 15:56:18 +0200 Subject: [PATCH 18/18] fix typo --- docs/_src/tutorials/tutorials/14.md | 2 +- tutorials/Tutorial14_Query_Classifier.ipynb | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/_src/tutorials/tutorials/14.md b/docs/_src/tutorials/tutorials/14.md index 98fed2995c..f0849c84ef 100644 --- a/docs/_src/tutorials/tutorials/14.md +++ b/docs/_src/tutorials/tutorials/14.md @@ -349,7 +349,7 @@ For example, we may be interested in detecting the sentiment or classifying the - Zero-shot classification is very versatile: by choosing a suitable base transformer, you can classify the text without any training dataset. You just have to provide the candidate categories. #### Using custom classification models -We can use a public model, available in the Hugging Face Hub. For example, if we want to classify the sentiment of the queries, we cam choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment. +We can use a public model, available in the Hugging Face Hub. For example, if we want to classify the sentiment of the queries, we can choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment. *In this case, the `labels` parameter must contain a list with the exact model labels. The first label we provide corresponds to output_1, the second label to output_2, and so on.* diff --git a/tutorials/Tutorial14_Query_Classifier.ipynb b/tutorials/Tutorial14_Query_Classifier.ipynb index 60ab0f7792..9401c2902c 100644 --- a/tutorials/Tutorial14_Query_Classifier.ipynb +++ b/tutorials/Tutorial14_Query_Classifier.ipynb @@ -602,7 +602,7 @@ "metadata": {}, "source": [ "#### Using custom classification models\n", - "We can use a public model, available in the Hugging Face Hub. For example, if we want to classify the sentiment of the queries, we cam choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.\n", + "We can use a public model, available in the Hugging Face Hub. For example, if we want to classify the sentiment of the queries, we can choose an appropriate model, such as https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment.\n", "\n", "*In this case, the `labels` parameter must contain a list with the exact model labels.\n", "The first label we provide corresponds to output_1, the second label to output_2, and so on.*" @@ -758,7 +758,7 @@ "provenance": [] }, "kernelspec": { - "display_name": "Python 3.7.13 ('venv': venv)", + "display_name": "Python 3.7.11 ('haystack-dev')", "language": "python", "name": "python3" }, @@ -772,11 +772,11 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.13" + "version": "3.7.11" }, "vscode": { "interpreter": { - "hash": "8db71796417881f0bc38f90aea8b7586b2d6898630e76f69c59e615595ffff43" + "hash": "a1c4180befe5334d9af26d84758dc08f43161c3b98a4eb4d4a43d7491d015a65" } } },