Linear Regression from Scratch in Python

🧠 Problem Statement

Implement Linear Regression from scratch without using libraries like Scikit-learn. The objective is to deeply understand how linear regression works under the hood by writing every step manually.

---

🎯 Goal

To build a minimal and fully transparent linear regression model using only NumPy, with support for:

• Prediction (y = mX + c)
• Model training using Gradient Descent
• Evaluation using MSE and R² Score
• Inspecting learned parameters (slope and intercept)

---

🚀 Why?

Many machine learning engineers and data scientists rely on libraries like Scikit-learn. However, understanding what happens behind the scenes—especially in a fundamental algorithm like linear regression—builds stronger intuition and improves problem-solving ability.

This project was implemented to build a crystal-clear understanding of:

• The math behind linear regression
• The mechanics of gradient descent
• The meaning and effect of loss functions like Mean Squared Error (MSE)

---

📐 Mathematical Intuition (Step-by-Step)

1. Equation of a Line

The hypothesis (predicted value) in linear regression is : y_pred = mX + c

Where:

• m is the slope (coefficient)
• c is the y-intercept

---

2. Error Calculation

The error is measured using Mean Squared Error (MSE): MSE = (1/n) * Σ(y_pred - y_actual)²

This tells us how far our predicted values are from the actual values.

---

3. Gradient Descent (Optimization)

To minimize error, we update m and c using the derivatives of the loss function (MSE):

Partial Derivatives:

∂MSE/∂m = (2/n) * Σ((y_pred - y_actual) * X) ∂MSE/∂c = (2/n) * Σ(y_pred - y_actual)

Update Rules:

m = m - learning_rate * ∂MSE/∂m c = c - learning_rate * ∂MSE/∂c

Repeat these updates for n_iterations.

---

🛠️ How to Use

from LinearModel import LinearModel

# Sample Data
X = [1, 2, 3, 4, 5]
y = [3, 4, 2, 5, 6]

# Model Initialization
model = LinearModel(lr=0.01, n_ittr=1000)

# Train Model
model.fit(X, y)

# Predict
predictions = model.predict([6, 7])
print(predictions)

# Evaluate
model.score(X, y)

# View Parameters
model.in_site()