[llm] Replace model_name with *required* base_model, add preset LLM registry, update internal adapter modules

[tgaddair]

Summary

The new base_model parameter is also just a string, backed by a anyOf schema that either validates whether the provided string is a Ludwig-supported preset through JSON or an otherwise valid model in the Huggingface repo at runtime (i.e. via validation checks).

It is the first required parameter in Ludwig, outside of input and output features, and it uses a bespoke schema tweak to enforce that. It is also the first parameter to use an anyOf implementation; this is necessary because oneOf validation requires a value to match exactly one option schema (a preset LLM name would partially match both schemas and cause an error).

If the user does choose a preset LLM then after JSON validation but before config object initialization this value will be swapped for the full slash-delimited path defined in the MODEL_PRESETS dictionary.

Usage

base_model: llama-7b

is valid and can be verified by JSON or by the validation checks. By the end of config object initialization it has been replaced by huggyllama/llama-7b (which is defined in MODEL_PRESETS).

base_model: bigscience/bloom-3b

is also valid but can only be verified during config-object initialization by the extra validation checks.

and

base_model:  <anything other than a string>

will cause an immediate validation failure.

Necessary follow-up PRs, prompted by changes here

• Looks like the required field is missing for model-level parameters (e.g. input_features). Probably an accidental regression. - Fix: int: Add back required for input and output features to the Ludwig JSON schema #3442
• Add official support for required parameters using marshmallow's in-built options
• Add support for arbitrary selector or "controller" fields (other than type).
• Investigate whether the extra metadata dicts inserted into marshmallow fields are necessary anymore

[github-actions]

Unit Test Results

  6 files  ±0    6 suites  ±0   1h 10m 6s ⏱️ - 6m 17s
33 tests ±0  29 ✔️ ±0    4 💤 ±0  0 ❌ ±0 
99 runs  ±0  87 ✔️ ±0  12 💤 ±0  0 ❌ ±0 

Results for commit 0eaa15f. ± Comparison against base commit f905b32.

♻️ This comment has been updated with latest results.

[arnavgarg1]

LGTM