Spam Link Detection System - SVM Approach

This machine learning project uses URL features to automatically detect whether a web page contains spam. The model developed is a Support Vector Machine (SVM), which evaluates the predictive performance on a dataset consisting of various URLs.

Dataset Overview

The dataset includes features derived from URLs to determine their spamminess:

• Full URL
• Domain name
• Path tokens
• Query parameters

The data undergoes preprocessing which includes parsing URLs, removing stopwords, and lemmatizing terms. The dataset is then vectorized using TF-IDF to prepare for SVM training.

Model Training and Evaluation

The SVM was trained with the following steps:

• The URLs were preprocessed to split into components.
• Features were extracted using TF-IDF vectorization.
• An SVM with default parameters was initially trained.
• Hyperparameter tuning was performed using grid search to optimize the model.

Results Summary

• The SVM achieved an accuracy of 95%.
• Detailed performance metrics include a precision of 94% for non-spam and 100% for spam, with corresponding recall rates of 100% and 81%.

Usage

To replicate the analysis:

• Ensure all dependencies are installed.
• Run the Python scripts to preprocess data, train the SVM, and evaluate its performance.
• The trained model can be used to predict the spamminess of new URLs.

For more details, see the provided Python scripts and Jupyter notebooks.

Contact

For inquiries or contributions, please contact spomar36@gmail.com.

Production Services

To ensure our spam detection system operates seamlessly in a production environment, we leverage the following cloud services and ML operations practices:

AWS

• Amazon S3: Store and retrieve any amount of data at any time.
• AWS Lambda: Run code in response to data changes in S3 buckets.
• Amazon SageMaker: Train, tune, and deploy machine learning models.

Azure

• Azure Blob Storage: For large-scale data storage.
• Azure Functions: Event-driven compute for data processing.
• Azure Machine Learning Service: Build, train, and deploy models.

GCP

• Google Cloud Storage: Durable and scalable object storage.
• Cloud Functions: Serverless execution of functions in response to cloud events.
• AI Platform: End-to-end machine learning model lifecycle management.

ML Operations

• Continuous Training: Implement pipelines to retrain models with new data.
• Data Validation: Use services like AWS Data Pipeline or Azure Data Factory for data integrity checks.
• Model Monitoring: Track model performance and data drift to maintain accuracy over time.

These services ensure our model stays updated, scalable, and maintainable, with robust ML operations that support continuous improvement.

Quickstart

git clone https://github.com/GOSS-hash/NLP-project