Text Analysis

This project provides a machine learning pipeline for sentiment analysis using a Logistic Regression (LR) model and Transformer-based models. The pipeline supports both model types, and the API allows users to predict sentiment on user-provided text data. The transformer model is more efficient in text analysis than LR model. Ensure fewer training epochs to prevent overfiiting.

Project Structure

The project is organized as follows:

• LR_model/: Contains the Logistic Regression (LR) model and pipeline.

  • clean_data.py: Script for data cleaning.
  • make_predictions.py: Script for making predictions.
  • model.py: Defines the Logistic Regression model.
  • preprocess.py: Preprocessing routines for the dataset.
  • read_data.py: Reads and loads the dataset.
  • run_pipeline.py: Runs the full pipeline, including data cleaning, model training, and evaluation.
  • EDA.ipynb: Exploratory Data Analysis for the dataset.

• transformers/: Contains the Transformer model for prediction.

  • Transformer_model.ipynb: Jupyter notebook for training and evaluating a transformer model.
  • predictions.py: Script for generating predictions from the transformer model.
  • run_predictions.py: Script to run predictions using the transformer model.

• api/: Contains the FastAPI application.

  • fastapi.app: The FastAPI application serving the prediction API.

• DockerFile.fastapi: Dockerfile to containerize the FastAPI app and model pipelines.

Requirements

Make sure to install the necessary dependencies. Use the provided requirements.txt in api/ to install dependencies:

pip install -r requirements.txt For Docker users, all dependencies are included in the Dockerfile.

Running the API Locally To run the API locally, follow the steps below.

1. Set up the environment Ensure that your environment is set up with the necessary dependencies. If you are not using Docker, follow these steps to set up the project:

Install the required dependencies

pip install -r /api/requirements.txt 2. Run the FastAPI App Navigate to the /api/ folder and run the FastAPI app using the following command:

Copy code uvicorn fastapi.app:app --reload --host 0.0.0.0 --port 8000 This will start the FastAPI app locally at http://localhost:8000.

3. Access the API Docs Once the server is running, you can access the auto-generated API documentation at:

Swagger UI: http://localhost:8000/docs

ReDoc: http://localhost:8000/redoc

4. Test the Model Predictions You can test the model by sending a POST request to /predict with the following format:

json

{ "text": "Your input text for sentiment analysis" } The response will be a sentiment prediction, including a label (Negative, Neutral, Positive) and a confidence score.

5. Running the LR Pipeline To run the full Logistic Regression pipeline, you can use the script run_pipeline.py in the LR_model/ folder:

Copy code python LR_model/run_pipeline.py This script will clean the data, train the model, and evaluate its performance.

6. Running the Transformer Model To run the transformer model and make predictions, use the run_predictions.py script in the transformers/ folder:

Copy code python transformers/run_predictions.py This will run the transformer model's prediction pipeline.

Docker Setup If you'd prefer to run the application in a Docker container, you can build and run the Docker image. The Dockerfile is located in DockerFile.fastapi.

1. Build the Docker Image From the project root directory, run the following command to build the Docker image:

Copy code docker build -f DockerFile.fastapi -t my-fastapi-app .

2. Run the Docker Container Once the image is built, you can run it with:

Copy code docker run -p 8000:8000 my-fastapi-app The FastAPI app will be available at http://localhost:8000 inside the Docker container.

Troubleshooting:

If you encounter any issues while running the API or model pipelines, please check the logs for detailed error messages. You can also refer to the FastAPI Documentation for further assistance.

Conclusion:

This project provides an end-to-end pipeline for sentiment analysis using both Logistic Regression and Transformer models. You can use the FastAPI application to serve the models and make predictions via API requests. Additionally, Docker support makes it easy to deploy and run the app in a containerized environment.

This README provides comprehensive instructions on how to set up, run, and test the application locally