Recommendation Systeme Project

INTRO

c'est projet a pour but d'optimiser le rendement d'un site e_commerce , grace a l'ajout d'un module python qui va aide le propriete du site a ameliore ton publication , a satisfaire leur client.

Project structure

├── config                       <- Directory containing configuration files
├── README.md                 <- The top-level README for developers using this project.
├── dataset                   <- Sample data from different sources for unit and integration tests
│   ├── external              <- Data from third party sources.
│   └── raw                   <- The original, immutable data dump.
│
├── docs               
│
├── models                    <- Trained and serialized models, model predictions, or model summaries
│
├── notebooks                 <- Jupyter notebooks. 
│
├── schema                    <- PartsIO Json Schema
│
├── reports                   <- Generated analysis as HTML, PDF, LaTeX, etc.
│   └── figures               <- Generated graphics and figures to be used in reporting
│
├── requirements.txt          <- The requirements 
│                         
│
├── setup.py                  <- Make this project pip installable with `pip install -e`
├── tests                     <- The pytest tests root directory for unit / integration / e2e tests
├── src/datascience_pio                
│   ├── __init__.py    
│   │
│   ├── util                  <- Scripts with common processing
│   │
│   ├── data                  <- Scripts to download or generate data
|   |   ├── __init__.py    
│   │   └── common 
|   |   |   |_ s3.py          <- script containes aws/s3 related operations to read data if needed
|   |   |   |_ pio_articles.py     <- contains pio articles intermidiate dataframe specification builder
│   │   └── kromeda             <- kromeda provider related readers 
|   |       |_ articles.py    <- script to read and map kromeda articles to pio Intermediate dataframe       
│   │
│   ├── process               <- Scripts to process pio intermidiate dataframes
|   |   ├── __init__.py    
│   │   └── catalog           <- model mapping scripts pio intermidiate dataframe to pio model
|   |       |_ build_articles.py  <- build articles
│   │   └── matching              <- matching scripts 
|   |       |_ match_vehicles_kromeda_tecdoc.py  <- script convert pio intermidiate dataframe to pio categories model
│   │
│   ├── iamodels              <- Scripts to train IA models and then use trained models to make
│   │   │                       predictions
|   |   ├── __init__.py    
│   │   ├── predict_model.py
│   │   └── train_model.py
│   ├── record                <- Scripts to write pio data into storage engines
|   |   ├── __init__.py    
│   │   ├── mongodb           <- directory for scripts to write pio objects to MongoDB
│   │   └── elasticsearch     <- directory for scripts to write Elasticsearch objects to MongoDB
│   └── visualization         <- Scripts to create exploratory and results oriented visualizations
│       └── visualize.py
│
└── tox.ini                   <- tox file with settings for running tox; see tox.testrun.org

Development

Prerequisites

All you need is the following configuration already installed:

• Git
• pyenv prerequisites for ubuntu. Check the prerequisites for your OS.

sudo apt-get update; sudo apt-get install make build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm \
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev

• pyenv installed and available in path pyenv installation with Prerequisites
• For tests/integration tests with mongoDB, you need Docker

Add format, lint code tools

Autolint/Format code with Black in IDE:

• Auto format via IDE https://github.com/psf/black#pycharmintellij-idea

• [Optional] You could setup a pre-commit to enforce Black format before commit https://github.com/psf/black#version-control-integration

• Or remember to type black . to apply the black rules formatting to all sources before commit

• Jenkins will complain and tests will fail if black format is not applied

• Add same mypy option for vscode in Preferences: Open User Settings

• Use the option to lint/format with black and flake8 on editor save in vscode

Checked optional type with Mypy PEP 484

Configure Mypy to help annotating/hinting type with Python Code. It's very useful for IDE and for catching errors/bugs early.

• Install mypy plugin for intellij
• Adjust the plugin with the following options:

  "--follow-imports=silent",
  "--show-column-numbers",
  "--ignore-missing-imports",
  "--disallow-untyped-defs",
  "--check-untyped-defs"
  

• Work in Progress to adjust Mypy preferences
• Documentation: Type hints cheat sheet (Python 3)
• Add same mypy option for vscode in Preferences: Open User Settings

Install Sonarlint plugin

Detect Code Quality and Security issues on the fly

• Use SonarLint in IntelliJ IDEA
• Use SonarLint in vscode

Isort

• isort is the default on pycharm
• isort with vscode
• Lint/format/sort import on save with vscode in Preferences: Open User Settings:

{
    "editor.formatOnSave": true,
    "python.formatting.provider": "black",
    "[python]": {
        "editor.codeActionsOnSave": {
            "source.organizeImports": true
        }
    }
}

• isort configuration for pycharm. See Set isort and black formatting code in pycharm
• You can use make lint command to check flake8/mypy rules & apply automatically format black and isort to the code with the previous configuration

isort . --virtual-env dspioenv

PROJECT SCOPE

Dans ce projet ,on utilise 4 algorithme de base pour genere le recommendation:

• Algorithme Populaire :

Base sur les articles le plus populaire

• Algorithme Similarity:

Base sur les articles le plus similaire

• Algorithme Contenu:

Base sur le contenu, c'est a dire a leur produit achete en avant

• Algorithme hybrid:

Combinaison de articles similaire et le contenu

Exemple and Integration