Classification: Implement Fine-Tuning Initiation Endpoint

[nishika26]

Description
Build an API endpoint to initiate fine-tuning of classification models. This will include handling data preparation, interaction with the OpenAI API, and ensuring the process runs asynchronously to support scalable operations.

Tasks

• Implement stratified splitting of the input dataset (support multiple split ratios: train/val/test).

• Convert each split into the required .jsonl format for OpenAI fine-tuning.

• Initiate fine-tuning jobs using the OpenAI API.

• Run fine-tuning as a background task using FastAPI's background task support.

• Write unit and integration tests to validate:

• Split logic correctness

• JSONL format structure

• Fine-tuning initiation flow

• Background task triggering

Context
This endpoint is a core component of the classification flow, enabling users to fine-tune OpenAI models with custom datasets and use the resulting models within NGO chatbot workflows.