Data Science Project Boilerplate

This boilerplate is designed to kickstart data science projects by providing a basic setup for database connections, data processing, and machine learning model development. It includes a structured folder organization for your datasets and a set of pre-defined Python packages necessary for most data science tasks.

Structure

The project is organized as follows:

• app.py - The main Python script that you run for your project.
• explore.py - A notebook to explore data, play around, visualize, clean, etc. Ideally the notebook code should be migrated to the app.py when moving to production.
• utils.py - This file contains utility code for operations like database connections.
• requirements.txt - This file contains the list of necessary python packages.
• models/ - This directory should contain your SQLAlchemy model classes.
• data/ - This directory contains the following subdirectories:
  • interin/ - For intermediate data that has been transformed.
  • processed/ - For the final data to be used for modeling.
  • raw/ - For raw data without any processing.

Setup

Prerequisites

Make sure you have Python 3.11+ installed on your. You will also need pip for installing the Python packages.

Installation

Clone the project repository to your local machine.

Navigate to the project directory and install the required Python packages:

pip install -r requirements.txt

Create a database (if needed)

Create a new database within the Postgres engine by customizing and executing the following command: $ createdb -h localhost -U <username> <db_name> Connect to the Postgres engine to use your database, manipulate tables and data: $ psql -h localhost -U <username> <db_name> NOTE: Remember to check the ./.env file information to get the username and db_name.

Once you are inside PSQL you will be able to create tables, make queries, insert, update or delete data and much more!

Environment Variables

Create a .env file in the project root directory to store your environment variables, such as your database connection string:

DATABASE_URL="your_database_connection_url_here"

Running the Application

To run the application, execute the app.py script from the root of the project directory:

python app.py

Adding Models

To add SQLAlchemy model classes, create new Python script files inside the models/ directory. These classes should be defined according to your database schema.

Example model definition (models/example_model.py):

from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String

Base = declarative_base()

class ExampleModel(Base):
    __tablename__ = 'example_table'
    id = Column(Integer, primary_key=True)
    name = Column(String)

Working with Data

You can place your raw datasets in the data/raw directory, intermediate datasets in data/interim, and the processed datasets ready for analysis in data/processed.

To process data, you can modify the app.py script to include your data processing steps, utilizing pandas for data manipulation and analysis.

Contributors

This template was built as part of the 4Geeks Academy Data Science and Machine Learning Bootcamp by Alejandro Sanchez and many other contributors. Find out more about 4Geeks Academy's BootCamp programs here.

Other templates and resources like this can be found on the school GitHub page.

📝 Instructions

Predicting the cost of health insurance for a person

The important insurance company 4Geeks Insurance S.L. wants to calculate, based on the physiological data of its customers what will be the premium (cost) to be borne by each of them. To do this, it has assembled a whole team of doctors, and based on data from other companies and a particular study, it has managed to gather a set of data to train a predictive model.

Step 1: Loading the dataset

The dataset can be found in this project folder under the name medical_insurance_cost.csv. You can load it into the code directly from the link:

https://raw.githubusercontent.com/4GeeksAcademy/linear-regression-project-tutorial/main/medical_insurance_cost.csv

Or download it and add it by hand in your repository. In this dataset, you will find the following variables:

1. age. Age of primary beneficiary (numeric)
2. sex. Gender of the primary beneficiary (categorical)
3. bmi. Body mass index (numeric)
4. children. Number of children/dependents covered by health insurance (numeric)
5. smoker. Is the person a smoker? (categorical)
6. region. Beneficiary's residential area in the U.S.: northeast, southeast, southwest, northwest (categorical)
7. charges. Health insurance premium (numerical)

Step 2: Perform a full EDA

This second step is vital to ensure that we keep the variables that are strictly necessary and eliminate those that are not relevant or do not provide information. Use the example Notebook we worked on and adapt it to this use case.

Be sure to conveniently divide the data set into train and test as we have seen in previous lessons.

Step 3: Build a linear regression model

You do not need to optimize the hyperparameters. Start by using a default definition, and improve it in the next step.

Step 4: Optimize the previous model

After training the model, if the results are not satisfactory, optimize it if possible.