Improve KServe compatibility

[varunsh-xilinx]

Summary of Changes

• Change STRING type to BYTES
• Add versioning support
• Change shape type from uint64 to int64

Closes #5

Motivation

Compatibility with the KServe is a key feature of the inference server.

Implementation

Changing STRING to BYTES is a nomenclature change. It could still use some implementation changes because internally it's dealt with as char as opposed to std::byte.

Versioning support is a more substantial feature. Versioning is only supported for modelLoad where it's loading from a repository. The other caveat is that if you load a versioned model, then you can only interact with the model with versioned APIs. The standard non-versioned APIs continue to exist. For versioned models, a version string is suffixed to the endpoint to allow for multiple versions of the model to be active at a time. Non-versioned models omit the version suffix entirely. The implementation uses an empty version string to indicate no version.

The KServe spec is inconsistent with its definition for the type of shape. In text, it says uint64 but the gRPC proto definition uses int64. Using int64 for shape allows the server to specify dynamic shape sizes (e.g. using -1920 indicates the shape could be up to 1920) for models that don't have fixed sizes.

Notes

N/A

[gbuildx]

Build successful!

[gbuildx]

Build successful!