[FEATURE] Quickstart example for sockeye

[michaelhochleitner]

Is your feature request related to a problem? Please describe.
I'm not sure how to run a sockeye (https://github.com/awslabs/sockeye) model with budgetml

Describe the solution you'd like
A quickstart example to run a sockeye model. For example the model built in https://awslabs.github.io/sockeye/tutorials/wmt.html .

Describe alternatives you've considered
Using https://github.com/jamesewoo/sockeye-serving/tree/master/src/sockeye_serving or writing FastAPI endpoints that import sockeye.

Additional context
https://github.com/jamesewoo/sockeye-serving/tree/master/src/sockeye_serving does not seem to be in active development.

[htahir1]

Hi @michaelhochleitner! I was not really aware of sockeye before your request. But from a brief glance into the repo and docs it seems it should be possible to pass sockeye in the requirements array and then just build a predictor that uses the MXNet model underneath to make a prediction. See the example for details on how to do it with HuggingFace (its the same logic really). You dont need to use sockeye_serving for this.

What would help to give a more concrete example is if you could give me the code to actually run inference on a trained sockeye model. How does it consume the data (What format?). How does it do the forward pass in the Transformer? Could you send over a script where you do that locally and then I can add an example in the repo to use BudgetML to deploy an API for it

[htahir1]

@michaelhochleitner Any update on this? Otherwise id have to close

[htahir1]

Closing due to inactivity