Plain C11 Win32 app running bert-base-NER, MiniLM and distilbart-mnli via the ORT C API (CPU + optional CUDA EP)

[aivrar (aivrar)]

I wanted to share a small project that uses the ONNX Runtime C API from plain C11 (no C++), in case it's useful to anyone doing the same on Windows.

It's a Win32 desktop SEO tool that runs three Hugging Face models locally through ORT:

• dslim/bert-base-NER for entity extraction. A WordPiece tokenizer written in C feeds input_ids / attention_mask / token_type_ids; argmax over the per-token logits, then ## subwords are merged back into entities.
• sentence-transformers/all-MiniLM-L6-v2 for embeddings. Attention-masked mean pooling over last_hidden_state (it falls back to output_0 for exports that name the output differently).
• valhalla/distilbart-mnli-12-3 for zero-shot classification. A GPT-2 byte-level BPE tokenizer in C (loading vocab.json + merges.txt); softmax over the 3 logits, reading entailment from index 2 (BART-MNLI order: contradiction, neutral, entailment).

A few things that worked well:

• One OrtEnv + OrtSessionOptions (ORT_ENABLE_ALL, 4 intra-op threads) shared by all sessions, with inputs passed via CreateTensorWithDataAsOrtValue over caller-owned int64_t buffers, so there's no extra copying.
• CPU and GPU builds from one binary path: the CUDA EP isn't linked. At runtime it does GetProcAddress(GetModuleHandleA("onnxruntime.dll"), "OrtSessionOptionsAppendExecutionProvider_CUDA") and falls back to CPU if the symbol is missing or the call fails. The CPU zip is 6 MB; the GPU zip ships the CUDA provider DLLs.
• Built against the prebuilt ORT 1.17.1 Windows package with CMake.

Source (MIT): https://github.com/aivrar/serp-to-prompt-writer. The ORT code is in src/onnx_nlp.c and src/nli.c.

Disclosure: I'm the author, and it was built with heavy AI coding assistance; I've run and tested it myself. Happy to answer questions about using ORT from C.