Fail validate when converting to numpy array in transformer

[eyalcha]

/kind bug

Issue #1972 removed np.array check and now conversion should be done in transformer (which is the correct approach).
The issue is that currently when converting to np.array it fails in validate function in transformer because it is not a list

kserve/python/kserve/kserve/model.py

Line 109 in aefac29

def validate(self, request):

[yuzisun]

@eyalcha transformer needs to send the request to predictor over the wire so I think it still needs to convert to a list according to the protocol right?

[eyalcha]

@yuzisun the issue is here :

kserve/python/kserve/kserve/model.py

Line 78 in c75e526

async def __call__(self, body, model_type: ModelType = ModelType.PREDICTOR):

If we need to convert to numpy array and then send to predict (with REST V1/V2), the validate function in transformer will fail.

import numpy as np

request = {
    "instances": [
      {}  
    ]
}
print(isinstance(request["instances"], list))
# True

request["instances"] = np.array(request["instances"])
print(isinstance(request["instances"], list))
# False

The assumption is that we need to convert the instances to array before sending it to predict. Before issue #1972 it was done in predictor, in the predict function with no validation. Now it was moved to pre process of transformer and the validate function fails because of the type change.

Am I missing something?

[yuzisun]

@eyalcha Can you provide a more detailed example for the issue ? As I understand sklearn predict is able to convert to either numpy or pandas data framework accordingly.

[yuzisun]

The validation is only done when the input is a dict type, you can pass the np array to the predict function.

kserve/python/kserve/kserve/model.py

Line 160 in 09f5f60

if isinstance(payload, Dict) and "instances" in payload and not isinstance(payload["instances"], list):