Skip to content

[Services] Add default probes if model is set #3522

@peterschmidt85

Description

@peterschmidt85

The following probes can be used by default if the user specified model but hasn't specified probes:

probes:
  - type: http
    method: post
    url: /v1/chat/completions
    headers:
      - name: Content-Type
        value: application/json
    body: |
      {
        "model": "<model property name>",
        "messages": [{"role": "user", "content": "hi"}],
        "max_tokens": 1
      }
    timeout: 30s

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions