Inference Flow

Inference flow

classify_query(query: str) is a synchronous local Python function executed after full fine-tuning; it is not a deployed API endpoint.

sequenceDiagram
    autonumber
    participant Caller as Local test query
    participant Fn as classify_query()
    participant Tok as AutoTokenizer
    participant M as Trained classifier
    Caller->>Fn: query: str
    Fn->>M: eval()
    Fn->>Tok: Encode query; max_length 128
    Tok-->>Fn: input_ids, attention_mask
    Fn->>Fn: Move tensors to model device
    Fn->>M: forward() under torch.inference_mode()
    M-->>Fn: 4 logits
    Fn->>Fn: softmax, argmax, ID2LABEL
    Fn-->>Caller: predicted_label, confidence, all_scores

Loading

The function places model in evaluation mode, tokenizes and truncates, moves tensors to the same device as the model, runs under torch.inference_mode(), computes a softmax, and returns:

{
  "query": "example text",
  "predicted_label": "billing",
  "confidence": 0.0,
  "all_scores": {"billing": 0.0, "technical": 0.0, "account": 0.0, "feature_request": 0.0}
}

The placeholder numeric values above are schema illustrations, not actual model predictions. The four probabilities produced by the real model sum to approximately one before rounding.