Multi-Channel Neural Network for Text Classification

This repository contains the implementation of a Multi-Channel Neural Network (MCNN-TC) for text classification, specifically designed for sentiment analysis on social media text. The model leverages a hybrid architecture that combines a Transformer, a Convolutional Neural Network (CNN), and a Recurrent Neural Network (RNN) to capture diverse textual features for improved classification accuracy.

The entire pipeline, from data preprocessing to model evaluation, is included. The project uses a PostgreSQL database to manage the preprocessed data and engineered features.

Model Architecture

The core of this project is the MultiChannelNN, which processes input text through three parallel channels to extract a rich set of features:

1. Transformer Channel: Utilizes a pre-trained bert-base-uncased model to generate contextualized word embeddings. The final hidden state of the [CLS] token is used as the high-level feature representation of the input text.

2. CNN Channel: Employs a 1D Convolutional Neural Network to capture local n-gram patterns. It processes word embeddings to identify key local phrases and motifs.

3. RNN Channel: Uses a bidirectional Gated Recurrent Unit (GRU) to model sequential patterns and long-range dependencies within the text.

The features extracted from these three channels are concatenated and passed through an attention mechanism to weigh their relative importance. Finally, a fully-connected layer performs the final classification.

Repository Structure

The project is organized into several Python scripts, each responsible for a specific part of the machine learning pipeline:

• data_preprocessing.py: Reads the raw train.csv data, performs text cleaning (removes URLs, mentions, special characters), corrects spelling, expands contractions, removes stopwords, and stores the results in a PostgreSQL database table named preprocessed_data.
• eda.py: Conducts exploratory data analysis on the preprocessed data from the database. It generates visualizations for text length distribution, keyword frequency, target class balance, and word clouds.
• feature_engineering.py: Generates additional features from the preprocessed text, including TF-IDF vectors and label-encoded categorical features. These are stored in a separate feature_data table.
• model_training.py: Defines the MultiChannelNN architecture. It fetches data, prepares it for the model, and executes the training loop. The trained model is saved as multi_channel_nn.pth.
• model_eval.py: Loads the trained model to perform inference on the test.csv dataset. It preprocesses the test data, generates predictions, and creates a submission.csv file in the format required for a Kaggle competition.

Installation

1. Clone the repository:

   git clone https://github.com/mohits-code/mcnn-tc.git
   cd mcnn-tc

2. Install dependencies: Ensure you have Python 3.8+ installed. You can install the required packages using pip:

   pip install torch transformers pandas psycopg2-binary nltk scikit-learn matplotlib seaborn wordcloud autocorrect contractions

3. Download NLTK data: Run the following in a Python interpreter to download necessary NLTK models:

   import nltk
   nltk.download('stopwords')
   nltk.download('punkt')

4. Set up PostgreSQL:

   • Install and run a PostgreSQL server.
   • Create a database.
   • Update the database connection details (DB_NAME, DB_USER, DB_PASS, DB_HOST, DB_PORT) at the top of the following files:
     • data_preprocessing.py
     • eda.py
     • feature_engineering.py
     • model_training.py

5. Download Data Files:

   • Place your train.csv and test.csv files in the root directory of the project.
   • Download the GloVe pre-trained word embeddings (glove.6B.300d.txt) from the Stanford NLP website and place the file in the root directory.

Usage

Follow these steps to run the complete pipeline.

1. Data Preprocessing

Execute the script to clean the training data and load it into your PostgreSQL database.

python data_preprocessing.py

2. Exploratory Data Analysis (Optional)

To gain insights into the dataset, run the EDA script. It will generate and display several plots.

python eda.py

3. Feature Engineering (Optional)

This script creates an alternative table with TF-IDF features. Note that the main training script model_training.py does not use this table by default.

python feature_engineering.py

4. Model Training

Train the MultiChannelNN model. The script will fetch data from the preprocessed_data table, train the model, and save the weights to multi_channel_nn.pth.

python model_training.py

5. Model Evaluation and Submission

Generate predictions on the test set. This script will use the saved multi_channel_nn.pth model and the test.csv file to create submission.csv.

python model_eval.py

License

This project is licensed under the MIT License. See the LICENSE file for details.