Skip to content

Replace segmentation model type heuristic with arch field#20515

Merged
TurboGit merged 1 commit intodarktable-org:masterfrom
andriiryzhkov:model_arch
Mar 14, 2026
Merged

Replace segmentation model type heuristic with arch field#20515
TurboGit merged 1 commit intodarktable-org:masterfrom
andriiryzhkov:model_arch

Conversation

@andriiryzhkov
Copy link
Contributor

Summary

  • Add arch field to dt_ai_model_info_t (backend layer), parsed from model config.json
  • Replace decoder output count heuristic in segmentation.c with explicit arch lookup ("sam2" or "segnext")
  • Models must declare "arch" in their config.json — missing arch fails with a clear error message

Motivation

The previous approach detected SAM vs SegNext by counting decoder outputs (3+ = SAM, 1 = SegNext). This is fragile and would break when adding new model architectures with different output counts. Using an explicit arch field makes model type detection reliable and extensible.

Copy link
Member

@TurboGit TurboGit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But then we need an update of data/ai_models.json with the new arch field, no?

Ok, found the PR on darktable-ai adding the arch field. I need to enable the notification for darktable-ai as I continuously fail to see the changes there :(

@TurboGit TurboGit added this to the 5.6 milestone Mar 14, 2026
@TurboGit TurboGit added priority: low core features work as expected, only secondary/optional features don't feature: redesign current features to rewrite labels Mar 14, 2026
@TurboGit
Copy link
Member

TurboGit commented Mar 14, 2026

@andriiryzhkov :

In the new models created I don't have the arch in config.json:

$ cat mask-object-sam21-small/config.json 
{
    "id": "mask-object-sam21-small",
    "name": "mask sam2.1 hiera small",
    "description": "Segment Anything 2.1 (Hiera Small) for interactive masking",
    "task": "mask",
    "backend": "onnx",
    "version": "1.0",
    "tiling": false
}

And then creating the mask fails, the message on the console:

14.1144 [segmentation] unknown arch '' for mask-object-sam21-small

If I add manually the arch into config.json all this working.

@andriiryzhkov
Copy link
Contributor Author

I see, missed to add in conversion script. Will update shortly

@andriiryzhkov
Copy link
Contributor Author

darktable-org/darktable-ai#9

Copy link
Member

@TurboGit TurboGit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@TurboGit TurboGit merged commit d26c3f8 into darktable-org:master Mar 14, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature: redesign current features to rewrite priority: low core features work as expected, only secondary/optional features don't

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants