Skip to content

Allow models to specify target framework versions #348

Description

@VivekPanyam

At export time, models should be able to specify version(s) of a framework that the model is compatible with.

When we load a model, Neuropod will attempt to find a backend that satisfies the specified range.

This combined with OPE will enable environments to provide many supported versions of each framework and have models choose which one to use.

This is useful in several cases:

  • If a model uses custom ops that are only compatible with one version of the framework (e.g. custom ops built against TF 1.15 headers vs TF 1.13.1 headers)
  • Being able to ensure that runtime and test environments are as similar as possible
  • Avoiding bugs in a specific version of the underlying framework

Target version ranges are specified as semver ranges (see https://semver.org/, https://docs.npmjs.com/misc/semver#ranges, https://docs.npmjs.com/misc/semver#advanced-range-syntax) which provides a very flexible way of defining the required framework.

TODO:

Once this is production ready, using dockerized OPE worker processes will also let us specify required CUDA versions (if any) and provide even more isolation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions