[ENH] Improved data preparation for LLM finetuning

[paxcema]

Description

This PR introduces some common data preparation utilities for fine-tuning LLMs that operate following the ChatCompletion format from OpenAI. Currently, these coupled versions of these methods are actively being used in our integrations for Anyscale and OpenAI (hence indirectly by LangChain, too).

Added tests for all of these, so that we can use them on any other LLM engines.

Type of change

• ⚡ New feature (non-breaking change which adds functionality). Refactor.

Verification Process

To ensure the changes are working as expected:

• Test Location: added unit tests, which pass.

Additional Media:

• I have attached a brief loom video or screenshots showcasing the new functionality or change.

Checklist:

• My code follows the style guidelines(PEP 8) of MindsDB.
• I have appropriately commented on my code, especially in complex areas.
• Necessary documentation updates are either made or tracked in issues.
• Relevant unit and integration tests are updated or added.

[dusvyat]

LGTM, see minor point

[pedrofluxa]

Left a comment on a hard-coded value. I am approving this PR as it should work as it is, but I strongly suggest to refactor the code a little bit.

[chandrevdw31]

Unable to Finetune as it says False

[paxcema]

@chandrevdw31 status is marked as generating in the screenshot above, and these finetune runs can easily take hours so this is normal. The model cannot be active before its status is complete.

If the user reports an actual error in the model record, please ask them to open a bug with it.