3. Setup Model Dependencies

This section contains information on how to setup the custom dependencies You would need to create 3 files for the custom preprocessing, postprocessing and the custom metrics for your model. Then place these files in a model_dependencies folder and provide the path in the configuration file.

Custom Preprocessing

'''
preprocess.py
input : input from the API endpoint
Define the function to preprocess the input before it goes to the model for inference.
The name of the preprocess function should be provided in the configuration file
'''
from register import PREPROCESS
import cv2
import numpy as np

@PREPROCESS.register_module(name='some_custom_preprocess_fxn')
def custom_preprocess_fxn(input):
  _channels = 3
  _input_shape = (224, 224)
  _channels_first = 1
  input = cv2.resize(
      input[0], dsize=_input_shape, interpolation=cv2.INTER_CUBIC)
  if _channels_first:
    input = np.reshape(input, (_channels, *_input_shape))
  else:
    input = np.reshape(input, (*_input_shape, _channels))
  return np.asarray(input, np.float32)

Custom Postprocessing

'''
postprocess.py
out : Output from the model
Define the function to postprocess the output of the model before it is returned via the API
The name of the postprocess function should be provided in the configuration file
'''
from register import POSTPROCESS

@POSTPROCESS.register_module(name='some_custom_postprocess_fxn')
def custom_postprocess_fxn(output):
  output = {'out': output[0],
            'probablity': output[1],
            'status': 200}
  return output

Custom Metrics

'''
custom_metrics.py
x : Output from the model
Define the function to create a custom metric to be logged into prometheus
The name of the metric should be provided in the configuration file
'''
from register import METRICS

@METRICS.register_module(name="custom_metric_1")
def cm1(x):
    return 1