mlserver

In this repo we have implemented example pipelines from three previous work and also some of our own imagined pipelines

1. Inferline
2. FA2
3. Scrooge

Also some pipelines were adaptod from industrial MLaaS products like:

1. Clarifai Workflows

Model sources:

audio and text models

Source:

1. HuggingFace

For image models

Source:

1. Timm
2. TorchVision

Selected variats family For each pipeline selected variants refer to its own READMD file. But in general following previous works like INFaaS, Cocktail, Morphling and MLPerf

INFaaS

1. Keras
2. PyTorch
3. GNMT

Model - Task - #Varaint alexnet densenet121 densenet169 densenet201 mobilenetv1 mobilenetv2 nasnetmobile resnet101 resnet101v2 resnet152 resnet152v2 resnet50 resnet50v2 resnext101 resnext50 vgg16 vgg19 inception-resnetv2 inceptionv3 xception nasnetlarge gnmt

Cocktail

Text Albert-base CodeBert DistilBert Albert-large XLNet Bert Roberta Albert-xlarge Albert-xxlarge

Image Classification Squeezenet MobileNEt Inception Resnet50 ResNet18 DenseNet-201 DenseNet-121 Xxception NasNet InceptionResnetV2

Morphling

Model type - model - # of models Img. Class.

ResNet (5), NASNet (2), VGG (2), Inception (2), DenseNet (1), MobileNet (2), EfficientNet (7),

Lang. Mod.

BERT (2), ALBERT (4), ELMo (1), NNLM (2), Small BERT (4), Word2vec (2), ELECTRA (2), Universal Sentence Encoder (4)