homework1/mt2022_hw1_data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-xclcis_1ELA"
   },
   "source": [
    "# Homework 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dbQuvrAB1ELN"
   },
   "source": [
    "**How to submit.** Sumbissions are through the [courses page](https://courses.cs.ut.ee/2022/mt/spring/Main/Homeworks). For this homework, submit this notebook with your code, examples and explanations. Make sure that the output is present in the version of the notebook that you submit. If you think some other files are necessary to understand your solutions, submit those files together with the notebook in a `.zip` file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Duym8aRn1ELO"
   },
   "source": [
    "## Looking into data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6rYMTnj_1ELP"
   },
   "source": [
    "In this homework, we will look into data that we can use for training translation models and into preprocessing steps that we went through during the second practice session.\n",
    "\n",
    "If you feel lost, review the practice session materials. Feel free to ask a question in Slack (channel #homeworks) if something is unclear."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WXkgRLn81ELP"
   },
   "source": [
    "### Task 1: Examining data (3 points)\n",
    "\n",
    "In the practice session we used the Europarl corpus to train our small translation model. Europarl contains translations of European Parliament proceedings. This corpus is good and quite clean, but it is relatively small and exhibits a very specific kind of language. In this task we will use the OpenSubtitles corpus, which is comprised of movie and TV subtitles.\n",
    "\n",
    "First, download the OpenSubtitles corpus. Choose a language pair where one language is English and another is some language that you know. Download the corpus fom here: http://opus.nlpl.eu/OpenSubtitles.php. Use the bottom-left triangle of the second table (the one where cells are colored green and pink) to download parallel files with plain text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "jPmhKv2h1ELQ"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zDHfaWz41ELR"
   },
   "source": [
    "**Question.** Which language pair did you choose?\n",
    "\n",
    "**Your answer:** ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FYuxYakD1ELR"
   },
   "source": [
    "**Subtask 1 (1 point)**. Print out 20 **randomly selected** sentence pairs from the corpus in a readable format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "LGYpmTNw1ELS"
   },
   "outputs": [],
   "source": [
    "### YOUR CODE ###"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MTfF8jnG1ELS"
   },
   "source": [
    "**Subtask 2 (2 points).** Real data is always messy, and it is important to look at your data and know what is there. How many of these 20 pairs are good translations of each other, how many are acceptable, how many are bad? Describe what kind of mismatches you see. Are there any repeating patterns?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f6cOLpZo1ELT"
   },
   "source": [
    "**Your answer:** ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VWSPt9Cl1ELT"
   },
   "source": [
    "### Task 2: Cleaning (3 points)\n",
    "\n",
    "During the practice session we applied very simple cleaning to our training corpus: we removed empty lines, sentences longer than 100 tokens and sentence pairs where one sentence has at least 9 times more tokens than the other. We assume that such sentence pairs are almost certainly bad (meaning that the two sentences in the pair are not good translations of each other)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0Jh5iPSx1ELU"
   },
   "source": [
    "**Subtask 1 (1 point).** Propose one more simple cleaning procedure for a parallel corpus. Your heuristic does not have to solve all our problems and remove all bad sentence pairs. It also does not have to have 0 false positive rate (if it might also remove some good sentence pairs, that's OK). It just has to be lightweight (no solutions involving neural networks) and intuitively seem like it would detect some bad translations.\n",
    "\n",
    "**Hint.** If you are stuck, try downloading the OpenSubtitles corpus in English and some language that you **don't** know. Look at some random sentence pairs. How do you guess if a pair is good or bad? Can you formalize it?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MzW7NnRu1ELV"
   },
   "source": [
    "**Your answer:** ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2OfdKZsn1ELV"
   },
   "source": [
    "**Subtask 2 (2 points).** Implement your idea. Apply your code to the OpenSubtitles corpus (if the corpus in the language pair of your choice is very big, you can process a portion of it). Did it work? If it did, bring some examples of bad sentence pairs that it removed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "B7eoAB4D1ELV"
   },
   "outputs": [],
   "source": [
    "### YOUR CODE ###"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "__pGFeqn1ELW"
   },
   "source": [
    "**Your answer:** ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f5oBXCH01ELW"
   },
   "source": [
    "### Task 3: Corpus statistics (2 points)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "26Epl8fq1ELW"
   },
   "source": [
    "We have used the length ratios of source and target sentences to filter out some bad sentence pairs. However, we cannot always expect the ratio to be close to 1. All languages are different, and some use more words than others to express the same meaning.\n",
    "\n",
    "In this task you will examine some corpus statistics. Let us look in more detail at a language pair that we will be dealing with in further homeworks: Estonian $\\leftrightarrow$ English. We will keep working with the Europarl corpus that we used in the second practice session."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "a6Yh_Yjg1ELX"
   },
   "source": [
    "**Subtask 1 (1 point).** Take the Estonian $\\leftrightarrow$ English Europarl corpus. Calculate how many tokens there are in the English side and in the Estonian side. (Tokenize the text beforehand. You can use any tokenizer: `mosestokenizer`, `spaCy`, or anything else.) Calculate how many **distinct** tokens (word types) there are in the English and in the Estonian side."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "4ztlssux1ELX"
   },
   "outputs": [],
   "source": [
    "### YOUR CODE ###"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dgPFgMjK1ELX"
   },
   "source": [
    "**Subtask 2 (1 point).** Plot the distribution of sentence lengths (measured in tokens) in the English and the Estonian side of the corpus. Compare the distributions. What can you say about the difference between English and Estonian languages based on these plots and the numbers that you calculated in subtask 1?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "JCwBOILk1ELY"
   },
   "outputs": [],
   "source": [
    "### YOUR CODE ###"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LsjRQU-D1ELY"
   },
   "source": [
    "**Your answer:** ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "V6N8_E991ELY"
   },
   "source": [
    "### Task 4: Subwords (2 points)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1WOzChCQ1ELZ"
   },
   "source": [
    "In this task, we will continue to use Estonian-English Europarl, but we will only need the English part.\n",
    "\n",
    "**Subtask 1 (2 points).** Train 2 SentencePiece models: one with vocabulary size 3,000 and the other with 10,000. Use the whole English side of the Europarl corpus for training. Split the texts with these two models. Look at the result. For each of the models, bring 5 examples of words which the model split in a way that seems intuitive to you, and 5 examples which look unintuitive. (An example of intuitive splitting could be `unbearable` $\\rightarrow$ `un bear able`.) Is the behaviour of two models different? If yes, how? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "CiRp9Cbe1ELZ"
   },
   "outputs": [],
   "source": [
    "### YOUR CODE ###"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sB9iLsIy1ELa"
   },
   "source": [
    "**Your answer:** ..."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "mt2022_hw1_data.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}