Can anyone help me train a NodeJS / JavaScript model ?

[bacloud22]

Hi,
I would like to train a super light-weight specialized model

[alleyxd]

For "light and specialized", a small coder model with a LoRA on your own examples works well in LlamaFactory. Something like Qwen/Qwen2.5-Coder-1.5B-Instruct (or 0.5B if you need it tiny).

1. Put your data in alpaca format, e.g. data/my_js.json:

   [{"instruction": "Write an Express route that ...", "input": "", "output": "const express = require('express') ..."}]

   A few hundred to a few thousand good pairs from your own codebase/task beats a big generic set.
2. Register it in data/dataset_info.json:

   "my_js": {"file_name": "my_js.json"}

3. Copy examples/train_lora/qwen3_lora_sft.yaml and change:

   model_name_or_path: Qwen/Qwen2.5-Coder-1.5B-Instruct
   template: qwen
   dataset: my_js
   output_dir: saves/coder-1.5b-js

4. llamafactory-cli train your_config.yaml. To try it, copy one of the configs in examples/inference/, set adapter_name_or_path to your output dir and run llamafactory-cli chat on it. llamafactory-cli export merges the LoRA into the base model.

A 1.5B LoRA like this trains fine on a single 12-16 GB GPU.