Text Complexity Identification 📚

Introduction

This repository houses the code I wrote for detecting the ease of readability of a given text excerpt which is the main task of the currently running CommonLit Readability Prize Competition

The main task in this competition is to take a text excerpt and predict it's ease of readability score (which is a continuous value).

I have also made a full training notebook in this competition.

Data

The main data consists of 2 important features (or columns): excerpt and target.

The excerpt is a basically a text sentence with a corresponding target value which denotes how "easy it is to read a sentence"

The data consist of 3 files: train.csv, test.csv and sample_submission.csv.

Training the Model

If you want to train the model on this data as-is, then you would typically have to perform 2 steps:

1. Getting the Data right

First, download the data from here.

Now, take the downloaded .zip file and extract it into a new folder: input/.

Make sure the input/ folder is at the same directory level as the train.py file.

2. Installing the dependencies

To run the code in this repository, you need a lot of frameworks installed on your system.

Make sure you have enough space on your disk and Internet quota before you proceed.

$ pip install -r requirements.txt

3. Training the Model

If you have done the above steps right, then just running the train.py script should not produce any errors.

To run training, open the terminal and change your working directory to the same level as the train.py file.

Now, for training do:

$ python train.py

This should start training in a few seconds and you should see a progress bar.

If you are having problems related to anything, please open an Issue and I will be happy to help!

I hope you found my work useful! If you did, then please ⭐ this repository!