Home

🚗 CarPricePro: Engineering Wiki

1. Project Architecture

The project follows a Microservices Architecture. The backend (FastAPI) handles the heavy lifting of mathematical transformations and model inference, while the frontend (Streamlit) provides an interactive interface for users.

2. Data Engineering Pipeline

Our pipeline uses a Pro-Level Transformation strategy to handle real-world messy data:

Robust Extraction: Uses Regular Expressions (Regex) to clean strings like "1,498 cc" or "$12,000 - $15,000".

KNN Imputation: Instead of filling missing values with zero, we use K-Nearest Neighbors to predict missing specs based on similar car profiles.

Quantile Clipping (Winsorization): To prevent the "Ferrari Effect," we cap the extreme 1% and 99% of data. This ensures the model isn't biased by hypercars.

3. Mathematical Optimization (Skewness)

Standard Log-Scaling often fails on car prices. We implement the Yeo-Johnson Power Transformer.

Objective: To achieve a Gaussian (Bell Curve) distribution.

Result: Reduced Price skewness from 17.65 to -0.009.

4. Feature Engineering & PCA

After One-Hot Encoding, the dataset reaches 70+ columns. We use PCA (Principal Component Analysis) to:

Compress high-dimensional category data (Brands/Fuel) into 2 components.

Reduce "noise" and prevent the model from overfitting.

Anomaly Detection: We use an Isolation Forest to tag and isolate cars that don't follow standard market patterns.

5. Model Performance

Algorithm: Random Forest Regressor (100 Estimators).

R² Score: 0.9147 (Explains 91.5% of price variance).

Cross-Validation: 5-fold CV shows high stability (+/- 0.017).

6. Deployment (DevOps)

The project is fully containerized using Docker.

Consistency: The train_columns.pkl file acts as the "API Contract" between the model and the UI, ensuring feature alignment.

Serialization: All transformers (Scaler, PCA, PowerTransformer) are serialized using joblib to prevent Feature Drift in production.