Main.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"Utilizzo.ipynb","provenance":[],"collapsed_sections":[]},"kernelspec":{"name":"python3","display_name":"Python 3"}},"cells":[{"cell_type":"code","metadata":{"id":"jXOzhlk_qBLz","executionInfo":{"status":"ok","timestamp":1640862774789,"user_tz":-60,"elapsed":121748,"user":{"displayName":"Giovanni Pilato","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"04534379576654093607"}},"outputId":"8eda53cb-dcf5-460d-f33a-45ff2d8ad02d","colab":{"base_uri":"https://localhost:8080/"}},"source":["from google.colab import drive\n","drive.mount('/content/drive')"],"execution_count":1,"outputs":[{"output_type":"stream","name":"stdout","text":["Mounted at /content/drive\n"]}]},{"cell_type":"code","metadata":{"id":"xWa6-WdOqGUL","executionInfo":{"status":"ok","timestamp":1640862941133,"user_tz":-60,"elapsed":7995,"user":{"displayName":"Giovanni Pilato","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"04534379576654093607"}}},"source":["%load_ext autoreload\n","%autoreload 2\n","from fastai import *\n","from fastai.text import *\n","from fastai.callbacks import *\n","from shutil import copyfile"],"execution_count":2,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"UUEDv1butDcG"},"source":["#Abstract"]},{"cell_type":"markdown","metadata":{"id":"AfWIAi3Qtkno"},"source":["Our goal is to obtain a language model fine tuned and from it obtaining a emotion classifier by text. In the end we will be able to generate infinite sentences with the labels assigned by the emotion classifier"]},{"cell_type":"markdown","metadata":{"id":"-ptZa15-wEWG"},"source":["#Dataset"]},{"cell_type":"markdown","metadata":{"id":"mRyjuMtYMJNP"},"source":["The datasets are two and they have been generated from open datasets:\n","\n","- The first has been generated from a Open Subititle's dataset, using in particular the italian and english languages.\n","- The second has been generated from the Ted's talks, always in italian and english, and it has been generated by web scraping the Ted's talks transcripts\n","\n","Later we used an english [emotion classifier](https://github.com/savan77/EmotionDetectionBERT), based on Bert technology, to classify the english sentences and obtain the predicted label that we use to build the italian dataset, by assegning them to the corresponding italian sentences.\n","\n","We also created a third dataset by merging the Sub and Ted datasets. "]},{"cell_type":"markdown","metadata":{"id":"Fow5w4ILqTcy"},"source":["#ULMFiT"]},{"cell_type":"markdown","metadata":{"id":"iaiLVal4qpDO"},"source":["Here we load the classifiers trained by [ULMFiT](https://arxiv.org/abs/1801.06146) technology based on a [previous work](https://medium.com/quantyca/ulmfit-for-italian-language-be31fa91e2c0)."]},{"cell_type":"code","metadata":{"id":"D4eObGOq5Wip","executionInfo":{"status":"ok","timestamp":1640862955110,"user_tz":-60,"elapsed":7954,"user":{"displayName":"Giovanni Pilato","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"04534379576654093607"}},"outputId":"557a7809-e56b-4af2-b13c-dd43c4deb746","colab":{"base_uri":"https://localhost:8080/","height":343}},"source":["#Models load\n","\n","learn_sub = load_learner('/content/drive/My Drive/Colab Notebooks (1)/ULMFiT Sub/', 'export.pkl')\n","learn_ted = load_learner('/content/drive/My Drive/Colab Notebooks (1)/ULMFiT Ted2/', 'export-ted2.pkl')\n","learn_merged = load_learner('/content/drive/My Drive/Colab Notebooks (1)/ULMFiT Merged/', 'export.pkl')"],"execution_count":3,"outputs":[{"output_type":"stream","name":"stderr","text":["/usr/local/lib/python3.7/dist-packages/torch/serialization.py:671: SourceChangeWarning: source code of class 'torch.nn.modules.loss.CrossEntropyLoss' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.7/dist-packages/torch/serialization.py:671: SourceChangeWarning: source code of class 'torch.nn.modules.sparse.Embedding' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.7/dist-packages/torch/serialization.py:671: SourceChangeWarning: source code of class 'torch.nn.modules.container.ModuleList' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.7/dist-packages/torch/serialization.py:671: SourceChangeWarning: source code of class 'torch.nn.modules.rnn.LSTM' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.7/dist-packages/torch/serialization.py:671: SourceChangeWarning: source code of class 'torch.nn.modules.container.Sequential' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.7/dist-packages/torch/serialization.py:671: SourceChangeWarning: source code of class 'torch.nn.modules.batchnorm.BatchNorm1d' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.7/dist-packages/torch/serialization.py:671: SourceChangeWarning: source code of class 'torch.nn.modules.dropout.Dropout' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.7/dist-packages/torch/serialization.py:671: SourceChangeWarning: source code of class 'torch.nn.modules.linear.Linear' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.7/dist-packages/torch/serialization.py:671: SourceChangeWarning: source code of class 'torch.nn.modules.activation.ReLU' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n"]},{"output_type":"display_data","data":{"text/html":[""],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{}},{"output_type":"display_data","data":{"text/html":[""],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{}},{"output_type":"display_data","data":{"text/html":[""],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{}},{"output_type":"display_data","data":{"text/html":[""],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{}},{"output_type":"display_data","data":{"text/html":[""],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{}},{"output_type":"display_data","data":{"text/html":[""],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{}}]},{"cell_type":"markdown","metadata":{"id":"mlzuUs2i1Oj0"},"source":["We use this method to preprocess the data and remove special characters that doesn't affect the sentiment analysis (i.e. dots, commas, ...)"]},{"cell_type":"code","metadata":{"id":"e6U3hgaw1M2W","executionInfo":{"status":"ok","timestamp":1640862959728,"user_tz":-60,"elapsed":342,"user":{"displayName":"Giovanni Pilato","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"04534379576654093607"}}},"source":["def preprocess(s):\n","  import re\n","  s = s.lower()\n","  s = re.sub(r\"[^a-zA-ZÀ-ú</>!?♥♡\\s\\U00010000-\\U0010ffff]\", ' ', s)\n","  s = re.sub(r\"\\s+\", ' ', s)\n","  s = re.sub(r'(\\w)\\1{2,}',r'\\1\\1', s)\n","  s = re.sub ( r'^\\s' , '' , s )\n","  s = re.sub ( r'\\s$' , '' , s )\n","  return s"],"execution_count":4,"outputs":[]},{"cell_type":"code","metadata":{"id":"j0HmitA64ZZN","executionInfo":{"status":"ok","timestamp":1640866686020,"user_tz":-60,"elapsed":2507,"user":{"displayName":"Giovanni Pilato","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"04534379576654093607"}},"outputId":"a41be259-2ed0-4677-aed7-1a3cd87b37cc","colab":{"base_uri":"https://localhost:8080/"}},"source":["#Sentences to test\n","\n","test_text = pd.DataFrame([\n","    'Questo è il giorno più felice della mia vita',\n","    'Penso di potermela cavare questa volta',\n","    'L\\'altro giorno sono andato a mare',\n","    'Domani andro al mare, che bello',\n","    'Credo di amarti',\n","    'Sei una persona orribile',\n","    'Vorrei viaggiare, sarebbe bellissimo',\n","    'Non voglio più uscire di casa',\n","    'La giornata oggi è pessima, non mi va di uscire',\n","    'Oggi sono malinconico, non mi va di uscire',\n","    'Mi sento triste, mi manca mio figlio',\n","    \"Mi manca mio figlio\"\n","])\n","\n","print('Labels:\\t',learn_merged.data.classes,'\\n')\n","\n","for s in test_text[0]:\n","  print('\\n\\nFrase: ',s)\n","  res = learn_sub.predict(preprocess(s))\n","  print('\\n\\tSub model: \\n\\t\\tPredicted label: ', res[0], '\\tProbabilities: ', res[2])\n","  res = learn_ted.predict(preprocess(s))\n","  print('\\n\\tTed model: \\n\\t\\tPredicted label: ', res[0], '\\tProbabilities: ', res[2])\n","  res = learn_merged.predict(preprocess(s))\n","  print('\\n\\tMerged model: \\n\\t\\tPredicted label: ', res[0], '\\tProbabilities: ', res[2])\n","\n"],"execution_count":13,"outputs":[{"output_type":"stream","name":"stdout","text":["Labels:\t ['anger', 'joy', 'neutral', 'optimism', 'pessimism', 'sadness'] \n","\n","\n","\n","Frase:  Questo è il giorno più felice della mia vita\n","\n","\tSub model: \n","\t\tPredicted label:  optimism \tProbabilities:  tensor([0.0189, 0.2521, 0.0747, 0.4743, 0.0511, 0.1289])\n","\n","\tTed model: \n","\t\tPredicted label:  optimism \tProbabilities:  tensor([0.0087, 0.1474, 0.1952, 0.4646, 0.0576, 0.1264])\n","\n","\tMerged model: \n","\t\tPredicted label:  joy \tProbabilities:  tensor([0.0037, 0.4122, 0.0781, 0.3479, 0.0423, 0.1158])\n","\n","\n","Frase:  Penso di potermela cavare questa volta\n","\n","\tSub model: \n","\t\tPredicted label:  optimism \tProbabilities:  tensor([0.0919, 0.1144, 0.3329, 0.3894, 0.0640, 0.0073])\n","\n","\tTed model: \n","\t\tPredicted label:  neutral \tProbabilities:  tensor([0.0156, 0.0770, 0.6636, 0.1331, 0.0602, 0.0506])\n","\n","\tMerged model: \n","\t\tPredicted label:  optimism \tProbabilities:  tensor([0.0298, 0.1212, 0.2287, 0.5359, 0.0721, 0.0123])\n","\n","\n","Frase:  L'altro giorno sono andato a mare\n","\n","\tSub model: \n","\t\tPredicted label:  neutral \tProbabilities:  tensor([0.0525, 0.1982, 0.3469, 0.1801, 0.1552, 0.0670])\n","\n","\tTed model: \n","\t\tPredicted label:  neutral \tProbabilities:  tensor([0.0101, 0.0346, 0.4815, 0.3753, 0.0555, 0.0431])\n","\n","\tMerged model: \n","\t\tPredicted label:  optimism \tProbabilities:  tensor([0.0227, 0.1902, 0.2247, 0.4212, 0.0987, 0.0424])\n","\n","\n","Frase:  Domani andro al mare, che bello\n","\n","\tSub model: \n","\t\tPredicted label:  joy \tProbabilities:  tensor([0.0323, 0.5608, 0.0793, 0.2537, 0.0263, 0.0477])\n","\n","\tTed model: \n","\t\tPredicted label:  neutral \tProbabilities:  tensor([0.0082, 0.0242, 0.5839, 0.3462, 0.0221, 0.0154])\n","\n","\tMerged model: \n","\t\tPredicted label:  optimism \tProbabilities:  tensor([0.0670, 0.2664, 0.0882, 0.4957, 0.0360, 0.0468])\n","\n","\n","Frase:  Credo di amarti\n","\n","\tSub model: \n","\t\tPredicted label:  optimism \tProbabilities:  tensor([0.0212, 0.2248, 0.2798, 0.3959, 0.0726, 0.0056])\n","\n","\tTed model: \n","\t\tPredicted label:  neutral \tProbabilities:  tensor([0.0043, 0.0394, 0.5940, 0.3241, 0.0286, 0.0096])\n","\n","\tMerged model: \n","\t\tPredicted label:  optimism \tProbabilities:  tensor([0.0368, 0.2224, 0.1299, 0.5295, 0.0596, 0.0218])\n","\n","\n","Frase:  Sei una persona orribile\n","\n","\tSub model: \n","\t\tPredicted label:  optimism \tProbabilities:  tensor([0.0307, 0.2498, 0.1810, 0.5204, 0.0094, 0.0087])\n","\n","\tTed model: \n","\t\tPredicted label:  neutral \tProbabilities:  tensor([0.0080, 0.0261, 0.5722, 0.3457, 0.0396, 0.0084])\n","\n","\tMerged model: \n","\t\tPredicted label:  optimism \tProbabilities:  tensor([0.0708, 0.2134, 0.2204, 0.4588, 0.0166, 0.0200])\n","\n","\n","Frase:  Vorrei viaggiare, sarebbe bellissimo\n","\n","\tSub model: \n","\t\tPredicted label:  optimism \tProbabilities:  tensor([0.0091, 0.1212, 0.0607, 0.7423, 0.0539, 0.0127])\n","\n","\tTed model: \n","\t\tPredicted label:  optimism \tProbabilities:  tensor([0.0031, 0.0408, 0.1390, 0.5835, 0.2169, 0.0168])\n","\n","\tMerged model: \n","\t\tPredicted label:  optimism \tProbabilities:  tensor([0.0037, 0.0731, 0.0312, 0.5713, 0.2942, 0.0265])\n","\n","\n","Frase:  Non voglio più uscire di casa\n","\n","\tSub model: \n","\t\tPredicted label:  pessimism \tProbabilities:  tensor([0.1073, 0.0567, 0.0913, 0.1468, 0.5388, 0.0590])\n","\n","\tTed model: \n","\t\tPredicted label:  optimism \tProbabilities:  tensor([0.0039, 0.0251, 0.3116, 0.3779, 0.2718, 0.0098])\n","\n","\tMerged model: \n","\t\tPredicted label:  pessimism \tProbabilities:  tensor([0.1089, 0.0430, 0.0988, 0.2247, 0.4779, 0.0468])\n","\n","\n","Frase:  La giornata oggi è pessima, non mi va di uscire\n","\n","\tSub model: \n","\t\tPredicted label:  pessimism \tProbabilities:  tensor([0.1464, 0.1910, 0.1061, 0.1059, 0.2537, 0.1969])\n","\n","\tTed model: \n","\t\tPredicted label:  pessimism \tProbabilities:  tensor([0.0127, 0.1391, 0.1608, 0.0785, 0.4944, 0.1144])\n","\n","\tMerged model: \n","\t\tPredicted label:  joy \tProbabilities:  tensor([0.1178, 0.2558, 0.2044, 0.2261, 0.1100, 0.0858])\n","\n","\n","Frase:  Oggi sono malinconico, non mi va di uscire\n","\n","\tSub model: \n","\t\tPredicted label:  joy \tProbabilities:  tensor([0.2673, 0.3016, 0.1067, 0.0770, 0.1379, 0.1095])\n","\n","\tTed model: \n","\t\tPredicted label:  pessimism \tProbabilities:  tensor([0.0246, 0.1843, 0.1614, 0.1001, 0.3683, 0.1613])\n","\n","\tMerged model: \n","\t\tPredicted label:  joy \tProbabilities:  tensor([0.1887, 0.2757, 0.1958, 0.1567, 0.1086, 0.0745])\n","\n","\n","Frase:  Mi sento triste, mi manca mio figlio\n","\n","\tSub model: \n","\t\tPredicted label:  sadness \tProbabilities:  tensor([0.0336, 0.1779, 0.0621, 0.1184, 0.0442, 0.5639])\n","\n","\tTed model: \n","\t\tPredicted label:  joy \tProbabilities:  tensor([0.0064, 0.4620, 0.0547, 0.1508, 0.1481, 0.1780])\n","\n","\tMerged model: \n","\t\tPredicted label:  sadness \tProbabilities:  tensor([0.0180, 0.3835, 0.0439, 0.0921, 0.0692, 0.3933])\n","\n","\n","Frase:  Mi manca mio figlio\n","\n","\tSub model: \n","\t\tPredicted label:  joy \tProbabilities:  tensor([0.0810, 0.2809, 0.1750, 0.1954, 0.0569, 0.2109])\n","\n","\tTed model: \n","\t\tPredicted label:  joy \tProbabilities:  tensor([0.0134, 0.4548, 0.0661, 0.2426, 0.0582, 0.1649])\n","\n","\tMerged model: \n","\t\tPredicted label:  sadness \tProbabilities:  tensor([0.0365, 0.2914, 0.0654, 0.1081, 0.0856, 0.4130])\n"]}]},{"cell_type":"markdown","metadata":{"id":"P1m8uQjN1Qke"},"source":["#Open Subtitle dataset"]},{"cell_type":"markdown","metadata":{"id":"zCrvxm6i_1v4"},"source":["Here we show the prediction probabilities for each previous test sentences with Open Subtitle model, and its confusion matrix."]},{"cell_type":"code","metadata":{"id":"IX_SWnoU0iHo","executionInfo":{"status":"ok","timestamp":1640865806926,"user_tz":-60,"elapsed":908,"user":{"displayName":"Giovanni Pilato","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"04534379576654093607"}},"outputId":"76add070-430a-449a-e152-d8c0dd0954e6","colab":{"base_uri":"https://localhost:8080/"}},"source":["print(learn_sub.data.classes,'\\n')\n","for s in test_text[0]:\n","  res = learn_sub.predict(preprocess(s))\n","  print(s,res[2])"],"execution_count":10,"outputs":[{"output_type":"stream","name":"stdout","text":["['anger', 'joy', 'neutral', 'optimism', 'pessimism', 'sadness'] \n","\n","Questo è il giorno più felice della mia vita tensor([0.0189, 0.2521, 0.0747, 0.4743, 0.0511, 0.1289])\n","Penso di potermela cavare questa volta tensor([0.0919, 0.1144, 0.3329, 0.3894, 0.0640, 0.0073])\n","L'altro giorno sono andato a mare tensor([0.0525, 0.1982, 0.3469, 0.1801, 0.1552, 0.0670])\n","Domani andro al mare, che bello tensor([0.0323, 0.5608, 0.0793, 0.2537, 0.0263, 0.0477])\n","Credo di amarti tensor([0.0212, 0.2248, 0.2798, 0.3959, 0.0726, 0.0056])\n","Sei una persona orribile tensor([0.0307, 0.2498, 0.1810, 0.5204, 0.0094, 0.0087])\n","Vorrei viaggiare, sarebbe bellissimo tensor([0.0091, 0.1212, 0.0607, 0.7423, 0.0539, 0.0127])\n","Non voglio più uscire di casa tensor([0.1073, 0.0567, 0.0913, 0.1468, 0.5388, 0.0590])\n","La giornata oggi è pessima, non mi va di uscire tensor([0.1464, 0.1910, 0.1061, 0.1059, 0.2537, 0.1969])\n","Oggi sono malinconico, non mi va di uscire tensor([0.2673, 0.3016, 0.1067, 0.0770, 0.1379, 0.1095])\n","Mi sento triste, mi manca mio figlio tensor([0.0336, 0.1779, 0.0621, 0.1184, 0.0442, 0.5639])\n"]}]},{"cell_type":"markdown","metadata":{"id":"NTl3-767-90u"},"source":["<a href=\"https://imgbb.com/\"><img src=\"https://i.ibb.co/NskXJ6L/Sub.png\" alt=\"Sub\" border=\"0\"></a>"]},{"cell_type":"markdown","metadata":{"id":"fV3afnd61Z4e"},"source":["#Ted dataset"]},{"cell_type":"markdown","metadata":{"id":"vHSpjwthASiv"},"source":["Here we show the prediction probabilities for each previous test sentences with Ted's talks model, and its confusion matrix."]},{"cell_type":"code","metadata":{"id":"2KtSoZ0c1mD7","executionInfo":{"status":"ok","timestamp":1640865812175,"user_tz":-60,"elapsed":782,"user":{"displayName":"Giovanni Pilato","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"04534379576654093607"}},"outputId":"04c4e6b2-2a87-4e29-d9e1-00726a13b2cc","colab":{"base_uri":"https://localhost:8080/"}},"source":["print(learn_ted.data.classes,'\\n')\n","for s in test_text[0]:\n","  res = learn_ted.predict(preprocess(s))\n","  print(s, res[2])"],"execution_count":11,"outputs":[{"output_type":"stream","name":"stdout","text":["['anger', 'joy', 'neutral', 'optimism', 'pessimism', 'sadness'] \n","\n","Questo è il giorno più felice della mia vita tensor([0.0087, 0.1474, 0.1952, 0.4646, 0.0576, 0.1264])\n","Penso di potermela cavare questa volta tensor([0.0156, 0.0770, 0.6636, 0.1331, 0.0602, 0.0506])\n","L'altro giorno sono andato a mare tensor([0.0101, 0.0346, 0.4815, 0.3753, 0.0555, 0.0431])\n","Domani andro al mare, che bello tensor([0.0082, 0.0242, 0.5839, 0.3462, 0.0221, 0.0154])\n","Credo di amarti tensor([0.0043, 0.0394, 0.5940, 0.3241, 0.0286, 0.0096])\n","Sei una persona orribile tensor([0.0080, 0.0261, 0.5722, 0.3457, 0.0396, 0.0084])\n","Vorrei viaggiare, sarebbe bellissimo tensor([0.0031, 0.0408, 0.1390, 0.5835, 0.2169, 0.0168])\n","Non voglio più uscire di casa tensor([0.0039, 0.0251, 0.3116, 0.3779, 0.2718, 0.0098])\n","La giornata oggi è pessima, non mi va di uscire tensor([0.0127, 0.1391, 0.1608, 0.0785, 0.4944, 0.1144])\n","Oggi sono malinconico, non mi va di uscire tensor([0.0246, 0.1843, 0.1614, 0.1001, 0.3683, 0.1613])\n","Mi sento triste, mi manca mio figlio tensor([0.0064, 0.4620, 0.0547, 0.1508, 0.1481, 0.1780])\n"]}]},{"cell_type":"markdown","metadata":{"id":"xOnkihtx_Gg7"},"source":["<a href=\"https://imgbb.com/\"><img src=\"https://i.ibb.co/VN9c2YJ/Ted2.png\" alt=\"Ted2\" border=\"0\"></a><br /><a target='_blank' href='https://it.imgbb.com/'></a><br />"]},{"cell_type":"markdown","metadata":{"id":"dhACDuNP1524"},"source":["#Merged dataset"]},{"cell_type":"markdown","metadata":{"id":"yDYaZAQ8AZMf"},"source":["Here we show the prediction probabilities for each previous test sentences with merged dataset model, and its confusion matrix."]},{"cell_type":"code","metadata":{"id":"r-_fNq4R2ZhK","outputId":"576bed38-6635-4e8e-cdb8-38381d557554","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1640865817128,"user_tz":-60,"elapsed":319,"user":{"displayName":"Giovanni Pilato","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"04534379576654093607"}}},"source":["print(learn_merged.data.classes,'\\n')\n","for s in test_text[0]:\n","  res = learn_merged.predict(preprocess(s))\n","  print(res[2])"],"execution_count":12,"outputs":[{"output_type":"stream","name":"stdout","text":["['anger', 'joy', 'neutral', 'optimism', 'pessimism', 'sadness'] \n","\n","tensor([0.0037, 0.4122, 0.0781, 0.3479, 0.0423, 0.1158])\n","tensor([0.0298, 0.1212, 0.2287, 0.5359, 0.0721, 0.0123])\n","tensor([0.0227, 0.1902, 0.2247, 0.4212, 0.0987, 0.0424])\n","tensor([0.0670, 0.2664, 0.0882, 0.4957, 0.0360, 0.0468])\n","tensor([0.0368, 0.2224, 0.1299, 0.5295, 0.0596, 0.0218])\n","tensor([0.0708, 0.2134, 0.2204, 0.4588, 0.0166, 0.0200])\n","tensor([0.0037, 0.0731, 0.0312, 0.5713, 0.2942, 0.0265])\n","tensor([0.1089, 0.0430, 0.0988, 0.2247, 0.4779, 0.0468])\n","tensor([0.1178, 0.2558, 0.2044, 0.2261, 0.1100, 0.0858])\n","tensor([0.1887, 0.2757, 0.1958, 0.1567, 0.1086, 0.0745])\n","tensor([0.0180, 0.3835, 0.0439, 0.0921, 0.0692, 0.3933])\n"]}]},{"cell_type":"markdown","metadata":{"id":"ekG9XSdK88FU"},"source":["<a href=\"https://imgbb.com/\"><img src=\"https://i.ibb.co/DCsVT0c/Merged.png\" alt=\"Merged\" border=\"0\"></a>"]},{"cell_type":"markdown","metadata":{"id":"rzGaWxMBSOxq"},"source":["\n","# Language models"]},{"cell_type":"markdown","metadata":{"id":"UPv_L5VZiP1p"},"source":["Using the language models we trained them how to speak italian. We have done it through our first language model based on the Wikipedia dataset and then we fine tuned it with our datasets, obtaining the last langauge model.<br>\n","The accuracy in pecentage of our model is a range of 20. In a way to obtain a good model we need a very large corpus, wich we have, but we need more time to obtain the italian datasets. This is going to be our next work"]},{"cell_type":"markdown","metadata":{"id":"IL3idFgWz-lg"},"source":["## TED's talks"]},{"cell_type":"code","metadata":{"id":"ftW23W1ZSOIT","executionInfo":{"status":"ok","timestamp":1600954554673,"user_tz":-120,"elapsed":2286,"user":{"displayName":"Giovanni Pilato","photoUrl":"","userId":"04534379576654093607"}},"outputId":"1212b91e-b87a-4368-b126-7c9a3491ce16","colab":{"base_uri":"https://localhost:8080/","height":267}},"source":["lm_ted = load_learner('/content/drive/My Drive/Colab Notebooks (1)/ULMFiT Ted2/', 'Language_model.pkl')"],"execution_count":null,"outputs":[{"output_type":"stream","text":["/usr/local/lib/python3.6/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.loss.CrossEntropyLoss' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.6/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.sparse.Embedding' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.6/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.container.ModuleList' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.6/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.rnn.LSTM' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.6/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.linear.Linear' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n"],"name":"stderr"},{"output_type":"display_data","data":{"text/html":[""],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{"tags":[]}},{"output_type":"display_data","data":{"text/html":[""],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{"tags":[]}}]},{"cell_type":"code","metadata":{"id":"irjxJ5Nmzvy3","executionInfo":{"status":"ok","timestamp":1600954627607,"user_tz":-120,"elapsed":948,"user":{"displayName":"Giovanni Pilato","photoUrl":"","userId":"04534379576654093607"}},"outputId":"eed06d98-a860-4d90-e784-44aa23a8be6b","colab":{"base_uri":"https://localhost:8080/","height":238}},"source":["# String prefix that has to be used for generated sentences\n","TEXT = 'Come'\n","# Number of words for each sentence to generate\n","N_WORDS = 5\n","# Number of sentences to generate\n","N_SENTENCES = 7\n","\n","print('\\n\\n'.join(lm_ted.predict(TEXT,N_WORDS) for _ in range(N_SENTENCES)))"],"execution_count":null,"outputs":[{"output_type":"stream","text":["Come terapie anni si minore di\n","\n","Come mette anno detto zona ]\n","\n","Come avanzate il ministro quattro ha\n","\n","Come della loro 35 da vaccino\n","\n","Come troppo alternative è un intendo\n","\n","Come quello un nel molte l'\n","\n","Come futuro l' fanteria . xxbos\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"ZarZK_qc8jVo"},"source":["## Subtitles"]},{"cell_type":"code","metadata":{"id":"Xvsdnasj8qbm","executionInfo":{"status":"ok","timestamp":1600954644262,"user_tz":-120,"elapsed":1872,"user":{"displayName":"Giovanni Pilato","photoUrl":"","userId":"04534379576654093607"}},"outputId":"f0488092-afb8-49c7-f696-085f82b154af","colab":{"base_uri":"https://localhost:8080/","height":267}},"source":["lm_sub = load_learner('/content/drive/My Drive/Colab Notebooks (1)/ULMFiT Sub/', 'Language-model-sub.pkl')"],"execution_count":null,"outputs":[{"output_type":"stream","text":["/usr/local/lib/python3.6/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.loss.CrossEntropyLoss' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.6/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.sparse.Embedding' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.6/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.container.ModuleList' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.6/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.rnn.LSTM' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.6/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.linear.Linear' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n"],"name":"stderr"},{"output_type":"display_data","data":{"text/html":[""],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{"tags":[]}},{"output_type":"display_data","data":{"text/html":[""],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{"tags":[]}}]},{"cell_type":"code","metadata":{"id":"LxAw52fI8qbq","executionInfo":{"status":"ok","timestamp":1600954647916,"user_tz":-120,"elapsed":974,"user":{"displayName":"Giovanni Pilato","photoUrl":"","userId":"04534379576654093607"}},"outputId":"ac733528-64b2-4ace-9d5a-f702a5ee3edd","colab":{"base_uri":"https://localhost:8080/","height":238}},"source":["# String prefix that has to be used for generated sentences\n","TEXT = 'come'\n","# Number of words for each sentence to generate\n","N_WORDS = 5\n","# Number of sentences to generate\n","N_SENTENCES = 7\n","\n","print('\\n\\n'.join(lm_sub.predict(TEXT,N_WORDS) for _ in range(N_SENTENCES)))"],"execution_count":null,"outputs":[{"output_type":"stream","text":["come la fondo di meta con\n","\n","come regolamento che la sapevi siete\n","\n","come e da portarla parecchio per\n","\n","come via un sto kit xxbos\n","\n","come sono dolore che dopo mio\n","\n","come per e settimana a via\n","\n","come molto ? xxbos contratto xxbos\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"WjSix2Cu_Bgt"},"source":["## Merged"]},{"cell_type":"code","metadata":{"id":"Gt9u9fPa_IGw","executionInfo":{"status":"ok","timestamp":1600954655876,"user_tz":-120,"elapsed":3020,"user":{"displayName":"Giovanni Pilato","photoUrl":"","userId":"04534379576654093607"}},"outputId":"d968af8e-4372-47f9-ae58-3e550d5ac74d","colab":{"base_uri":"https://localhost:8080/","height":267}},"source":["lm_merged = load_learner('/content/drive/My Drive/Colab Notebooks (1)/ULMFiT Merged/', 'Language-model-merged.pkl')"],"execution_count":null,"outputs":[{"output_type":"stream","text":["/usr/local/lib/python3.6/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.loss.CrossEntropyLoss' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.6/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.sparse.Embedding' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.6/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.container.ModuleList' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.6/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.rnn.LSTM' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n","/usr/local/lib/python3.6/dist-packages/torch/serialization.py:649: SourceChangeWarning: source code of class 'torch.nn.modules.linear.Linear' has changed. you can retrieve the original source code by accessing the object's source attribute or set `torch.nn.Module.dump_patches = True` and use the patch tool to revert the changes.\n","  warnings.warn(msg, SourceChangeWarning)\n"],"name":"stderr"},{"output_type":"display_data","data":{"text/html":[""],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{"tags":[]}},{"output_type":"display_data","data":{"text/html":[""],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{"tags":[]}}]},{"cell_type":"code","metadata":{"id":"9Iinzt7l_IG0","executionInfo":{"status":"ok","timestamp":1600954658761,"user_tz":-120,"elapsed":1298,"user":{"displayName":"Giovanni Pilato","photoUrl":"","userId":"04534379576654093607"}},"outputId":"1431da8f-8810-4d7e-da39-41c6d3a93d45","colab":{"base_uri":"https://localhost:8080/","height":238}},"source":["# String prefix that has to be used for generated sentences\n","TEXT = 'come'\n","# Number of words for each sentence to generate\n","N_WORDS = 5\n","# Number of sentences to generate\n","N_SENTENCES = 7\n","\n","print('\\n\\n'.join(lm_merged.predict(TEXT,N_WORDS) for _ in range(N_SENTENCES)))"],"execution_count":null,"outputs":[{"output_type":"stream","text":["come margaret ancora spiegare ? xxbos\n","\n","come ho voglio tutti tutti xxbos\n","\n","come vai xxbos imparano aveva xxbos\n","\n","come è questo po nella vita\n","\n","come non ci noi resto guardie\n","\n","come su hai organizzare andrebbe ad\n","\n","come mi togli del costo ?\n"],"name":"stdout"}]}]}