-
Notifications
You must be signed in to change notification settings - Fork 0
Home
The project follows a Microservices Architecture. The backend (FastAPI) handles the heavy lifting of mathematical transformations and model inference, while the frontend (Streamlit) provides an interactive interface for users.
Our pipeline uses a Pro-Level Transformation strategy to handle real-world messy data:
Robust Extraction: Uses Regular Expressions (Regex) to clean strings like "1,498 cc" or "$12,000 - $15,000".
KNN Imputation: Instead of filling missing values with zero, we use K-Nearest Neighbors to predict missing specs based on similar car profiles.
Quantile Clipping (Winsorization): To prevent the "Ferrari Effect," we cap the extreme 1% and 99% of data. This ensures the model isn't biased by hypercars.
Standard Log-Scaling often fails on car prices. We implement the Yeo-Johnson Power Transformer.
Objective: To achieve a Gaussian (Bell Curve) distribution.
Result: Reduced Price skewness from 17.65 to -0.009.
After One-Hot Encoding, the dataset reaches 70+ columns. We use PCA (Principal Component Analysis) to:
Compress high-dimensional category data (Brands/Fuel) into 2 components.
Reduce "noise" and prevent the model from overfitting.
Anomaly Detection: We use an Isolation Forest to tag and isolate cars that don't follow standard market patterns.
Algorithm: Random Forest Regressor (100 Estimators).
R² Score: 0.9147 (Explains 91.5% of price variance).
Cross-Validation: 5-fold CV shows high stability (+/- 0.017).
The project is fully containerized using Docker.
Consistency: The train_columns.pkl file acts as the "API Contract" between the model and the UI, ensuring feature alignment.
Serialization: All transformers (Scaler, PCA, PowerTransformer) are serialized using joblib to prevent Feature Drift in production.