Passing stopping_criteria to DeepSpeed MII

[tokestermw]

Hi, would it be possible to pass in a stopping_criteria inside .generate()?

https://huggingface.co/docs/transformers/main_classes/text_generation#transformers.generation_utils.GenerationMixin.generate

...
mii_generator = mii.mii_query_handle('name')
mii_generator.query({"query": ['hello']}, stopping_criteria=[])

Currently we get an error (can't pass a list of objects through grpc):

~/venv/lib/python3.7/site-packages/mii/utils.py in kwarg_dict_to_proto(kwarg_dict)
    176         return proto_value
    177
--> 178     return {k: get_proto_value(v) for k, v in kwarg_dict.items()}
    179
    180

~/venv/lib/python3.7/site-packages/mii/utils.py in <dictcomp>(.0)
    176         return proto_value
    177
--> 178     return {k: get_proto_value(v) for k, v in kwarg_dict.items()}
    179
    180

~/venv/lib/python3.7/site-packages/mii/utils.py in get_proto_value(value)
    173     def get_proto_value(value):
    174         proto_value = mii.grpc_related.proto.modelresponse_pb2.Value()
--> 175         setattr(proto_value, dtype_proto_field[type(value)], value)
    176         return proto_value
    177

KeyError: <class 'list'>

Use case is, for a text-generation task, I'd like to stop at a newline / custom token.

[mrwyattii]

This is not currently possible, but we are working on a solution to this. #109 (comment)

[tokestermw]

Got it thanks!