machine-learning-rest-api

Generic RESTful API for exported machine learning models.

Overview

This project creates an HTTP endpoint to interface with a pre-trained BERT question answering LLM. Currently this API provides a single GET request to receive the machine learning models response to a question.

This code is meant to be built upon to have additional HTTP endpoints that communicate with additional models added as needed for different machine learning functionality.

Dependencies

• Dotnet 6 (see csproj file for related packages: BERTTokenizers, OnnxRuntime, etc.)
• Python 3 (huggingface transformers lib, pytorch torch lib)
• Azure CLI

Process

1. Run python script. python3 CreateBERTmodel.py

2. Run C#/.NET API BertChat.cs

3. Call function locally, curl command below: curl http://localhost:7071/api/BertChat?question=I used to live in Vancouver BC in 2019. Where did I live four years ago?

Above the question can be anything. An example question is already added: I used to live in Vancouver BC in 2019. Where did I live four years ago?

BERT

Bidirectional Encoder Representations from Transformers (BERT) is a language model based on the transformer architecture, notable for its dramatic improvement over previous state of the art models. Wiki

I chose this model primarily because I wanted to learn more about it and I thought an API would be a great way to interact with the huggingface library.

Onnx

ONNX (Open Neural Network Exchange) is an open format for representing machine learning models. It allows for interoperability between different deep-learning frameworks and tools. Source

Specifically Onnx is used in this project because of the interaction between pytorch and C#/.NET code. It is not possible to train/create a model in C# so pytorch must be used. However, C#/.NET and Azure provide useful microservices needed for API development and microservice architecture. Onnx allows us to export a model in pytorch and invoke/interface with that model in our dotnet API.

Deplyoment

Although the deployment process is still in progress. The below command is to delpoy C#/.NET API code to Azure. You be logged into an Azure account via CLI and have a created function app by the name of machine-learning-rest-api.

func azure functionapp publish machine-learning-rest-api

Infrastructre as Code

Still to be created. I will either be using Kubernettes or ARM (Azure Resource Management). This deployment will specifically be for the microservices used in this project. For example the Function App machine-learning-rest-api deployment will be automated and configured via a script rather than manually in the Azure portal.

Sources

• Various ways to export python models.
• Tutorial on using BERTTokenizers and OnnxRuntime in C#.
• Toy example of interfacing with huggingfaces pre-trained chatbot DialoGPT