Skip to content

OpenAI endpoint stage 07 — embeddings and capabilities

Choose a tag to compare

View all tags
@TaewoooPark TaewoooPark released this 06 Jul 08:26
openai-endpoint-stage-07
0f65f3f

Stage 7 of the OpenAI-compatible endpoint branch.

Implemented:

  • Added model capability metadata to /v1/models for chat, completion, and embedding support.
  • Added POST /v1/embeddings with OpenAI-shaped embedding responses and usage accounting.
  • Added embedding input normalization for strings, string arrays, token-id arrays, and arrays of token-id arrays.
  • Added encoding_format: float and encoding_format: base64 support.
  • Added a non-thin DRIFT embedding implementation that routes the prompt through the existing split pipeline and pools the final hidden state.
  • Added explicit unsupported_embeddings errors for modes such as thin-head where hidden-state embeddings cannot be produced at the head.
  • Added tests for model capabilities, float embeddings, base64 embeddings, token-id embedding inputs, and unsupported embedding mode errors.

Verification:

  • python3 -m pytest tests/test_openai_api.py tests/test_sampling_controls.py -q (20 passed, 1 skipped because local Python lacks torch)
  • python3 -m compileall drift tests
Assets 2
Loading