[FEATURE] Implement export service

[inclinedadarsh]

Add Export Service for Fine-tuned Gemma Models

Fixes #43

This PR adds a new export service that allows users to export their completed training jobs in various formats.

Features

• Adapter Export: Returns existing adapter paths from the database (no processing needed)
• Merged Model Export: Merges adapters with base models and uploads to GCS
  • Supports both Unsloth and HuggingFace providers
  • Automatically detects backend provider from base model ID
  • Uploads to gs://gemma-export-bucket/merged_models/{job_id}/
  • Includes HF authentication for gated models
• GGUF Export: Convert merged model using llama.cpp.
• Security: Validates job ownership and completion status
• Automatic Cleanup: Removes temporary files after successful upload
• Firestore Integration: Updates job metadata with export paths

[inclinedadarsh]

Now this PR Fixes #31 as well.

GGUF Export Flow:

flowchart TD
    A[User requests GGUF export] --> B{GGUF in Firestore?}
    B -- Yes --> C[Return GGUF path]
    B -- No --> D{Merged model in Firestore?}
    D -- Yes --> E[Download merged model]
    E --> F[Convert to GGUF]
    F --> G[Push GGUF to bucket]
    G --> H[Update Firestore with GGUF]
    H --> I[Cleanup temp files]
    I --> J[Return GGUF path]

    D -- No --> K{Adapters in Firestore?}
    K -- No --> L[Return Error]
    K -- Yes --> M[Merge base model + adapters]
    M --> N[Upload merged model to bucket]
    N --> O[Update Firestore with merged model]
    O --> P[Use local merged model to convert GGUF]
    P --> Q[Push GGUF to bucket]
    Q --> R[Update Firestore with GGUF]
    R --> S[Cleanup merged model + GGUF]
    S --> T[Return GGUF path]

Loading

[inclinedadarsh]

So the limit is 16GB.

Step #2: ERROR: (gcloud.run.deploy) spec.template.spec.containers[0].resources.limits.memory: Invalid value specified for container memory. For 4.0 CPU, memory must be between 2Gi and 16Gi inclusive.
Step #2: For more troubleshooting guidance, see https://cloud.google.com/run/docs/configuring/memory-limits
Finished Step #2
ERROR
ERROR: build step 2 "gcr.io/cloud-builders/gcloud" failed: step exited with non-zero status: 1

[supreme-gg-gg]

Does this quantisation work on both unsloth and HF models because there might be nuances with quantisation. Plus, can we let the user configure quantisation because the training service supports choosing gguf_quantization from a list of 4-5 common options. We should definitely support q4_k quants for example.

[inclinedadarsh]

I tried doing this on google colab. The issue is the training method, model and provider heavly affect which quantizatiosn are available.

We'll have to first curate a permutation of all the quantizations available according to which options were selected. I guess that can be a different PR altogether, focusing on research main.

I did try the q8_0 quantization on both type of models, and they worked. However I'm not sure if models of different sizes will change this behaviour. I hope not.

[supreme-gg-gg]

Since if the job contains the export type already we dont want to create a new one, make sure the frontend displays the existing export options already and disallow the user to reselect an option that exists (since the frontend has the job data as well)

[supreme-gg-gg]

I think the following two features would be cool to add:

1. Pushing to HF space: I know you've mentioned this before but it would be really helpful if people can just export it to HF space instead GCS since it makes deployment a lot easier, also this seems like the original space to convert to GGUF from HF repo: https://huggingface.co/spaces/ggml-org/gguf-my-repo

2. Downloading GCS models (this is more like frontend + add another util on backend)? if I've fine tuned + exported to GGUF, I would want to run locally, perhaps for merged model / adapter too. The current way: only supports single file (GGUF, not anything else) + requires the export bucket to be public which is a bad idea since someone can technically download anyone else's model... A better way is to find how to best download assets from GCS (using a presigned URL I believe) and replace both frontend and backend for download with that.

However, if the current service already works they can be in a separate PR or a future feature, but both are tied with deployment so might as well ship them? #28

[inclinedadarsh]

Currently while testing merging the model, there seems to be an issue I'm not able to fix. At this point I have zero clue what might have caused this.

Testing information:

Model I was trying to merge: training_2c1a4996_gemma-3-270m-it-unsloth-bnb-4bit_531e25ee

Error:

2025-09-02 20:23:38,964 - ERROR - Failed to merge model for job training_2c1a4996_gemma-3-270m-it-unsloth-bnb-4bit_531e25ee: Unsloth: Saving LoRA finetune failed since # of LoRAs = 126 does not match # of saved modules = 0. Please file a bug report!

In case you're wondering, here's the input request payload:

{
  "job_id": "training_2c1a4996_gemma-3-270m-it-unsloth-bnb-4bit_531e25ee",
  "export_type": "merged",
  "hf_token": "TOKEN_HERE"
}

You can reproduce this issue simply by heading to our export service at https://export-service-10987549752.us-central1.run.app/docs. The application is live right now.

My investigation:

I didn't found anything in the docs or from AI. There's this GitHub issue I found, but it has 0 help as well: unslothai/unsloth#2238

Moreover, I also found this Discord thread, but at the end, they're just updating unsloth, so I'm not sure if it'll work.

You can try to upgrade and rebuild the image though.

[inclinedadarsh]

@supreme-gg-gg can you investigate in this?

[supreme-gg-gg]

Currently while testing merging the model, there seems to be an issue I'm not able to fix. At this point I have zero clue what might have caused this.

Testing information:

Model I was trying to merge: training_2c1a4996_gemma-3-270m-it-unsloth-bnb-4bit_531e25ee

Error:

2025-09-02 20:23:38,964 - ERROR - Failed to merge model for job training_2c1a4996_gemma-3-270m-it-unsloth-bnb-4bit_531e25ee: Unsloth: Saving LoRA finetune failed since # of LoRAs = 126 does not match # of saved modules = 0. Please file a bug report!

In case you're wondering, here's the input request payload:

{
  "job_id": "training_2c1a4996_gemma-3-270m-it-unsloth-bnb-4bit_531e25ee",
  "export_type": "merged",
  "hf_token": "TOKEN_HERE"
}

You can reproduce this issue simply by heading to our export service at https://export-service-10987549752.us-central1.run.app/docs. The application is live right now.

My investigation:

I didn't found anything in the docs or from AI. There's this GitHub issue I found, but it has 0 help as well: unslothai/unsloth#2238

Moreover, I also found this Discord thread, but at the end, they're just updating unsloth, so I'm not sure if it'll work.

You can try to upgrade and rebuild the image though.

I have never seen this before, no idea what's going on... I recommend trying with another training job / create a new simple job, or if it doesn't work just test on colab. i think you should report to unsloth probably if it still errors since GGUF seems to be something they actively work on?

[inclinedadarsh]

Still getting this issue on a different job.

2025-09-03 19:25:55,222 - ERROR - Failed to merge model for job training_sql-train-eval-dataset_gemma-3-1b-it-unsloth-bnb-4bit_e13f1731: Model merging failed: Unsloth: Saving LoRA finetune failed since # of LoRAs = 182 does not match # of saved modules = 0. Please file a bug report!

Will go ahead and report the bug to Unsloth, and do some experiments on google colab.

[supreme-gg-gg]

@inclinedadarsh I think we can merge the export service with training service? It seems to only contain two endpoints and they are very short lightweight executions, might be easier to manage if we put them together + they're related and pretty much identical in structure