Product catalog generator

This repo is the source code for a custom LLM and VLM fine tuned on LLama 2 and Llava1.5 based on Basalam products to infer enitty (product types) and attributes based on product data. You can use it on any similar dataset.

Datasets

Dataset V1 generated using GPT-3.5

GPT-3.5 generated product data

Dataset V2 generated using GPT-4

GPT-4 generated product data

Dataset for Vision catalog generated using GPT-4 (✅New)

GPT-4 generated product data

Models

Sft model version 1 based on llama 2 and GPT-3.5 data.

Model V1

Sft model version 2 based on llama 2 and GPT-4 data.

Model V2

Sft model based on Llava1.5 and GPT-4 data (✅New)

Model

LoRA file for Vision Catalog

Model

Evaluation

model	train loss	val loss	download
Model V1	0.07	0.08	Sft model version 1 based on llama 2 and GPT-3.5 data.
Model V2	0.1	0.12	Sft model version 2 based on llama 2 and GPT-4 data.
vision	0.11	0.13	Sft model based on llava1.5 and GPT-4 data.

Motivations

Problem definition and roadmap to solve it (in Persian). Virgool link.

How to use

Train:

To finetune a new model, you can either create a new YAML configuration file with your specific parameters or modify an existing one. You'll find example configuration files in the src/train/ directory (**config, **config). The default base model is NousResearch/Llama-2-7b-chat-hf, but you are free to change it. It's advisable to adjust the LoRA parameters accordingly if you do. Tailor other parameters to the needs of your task and dataset.

To initiate finetuning, navigate to the src directory and start the process with:

python -m train.train_wrapper --version v1

Here, --version v1 corresponds to the version of the finetuning configuration, which should match the name of your YAML file.

The training process includes several steps:

1- Parameter Initialization: Loads parameters from the specified YAML file.
2- Dataset Loading:
    - Retrieves the dataset from the Hugging Face hub using the _create_datasets_ function.
3- Training Execution: Handled by the _run_training_ method from the **[training_module](https://github.com/basalam/product-catalog-generator/blob/main/src/train/training.py):
    - Initializes configuration settings.
    - Prepares the model and tokenizer.
    - Sets up training arguments.
    - Begins the training cycle.
    - After completing the last iteration, saves the model, merging LoRA configurations with the base model using peft.
    - Uploads the trained model to the Hugging Face hub.
    - Concludes the process!

This structured approach ensures comprehensive management and execution of the model training process.

Inference:

For inference we use llm inference engine vllm.

Inference config such as model, prompt and response templates are located at BASE_DIR/configs/inference/. Start by running inference_wrapper in inference directory. The process is as follows:

1. Reading config from config file
2. Running inference_model from vllm_engine module
   • Args is read
   • LLM inference engine is built
   • For each sample input (prompt + input values (typically a product information)), a response is generated