Bug: REST API available for multilingual_universal_encoder

[stojanov-igor]

What happened?

I am trying to use the model from: hub/examples/colab/cross_lingual_similarity_with_tf_hub_multilingual_universal_encoder.ipynb

I intend to use the saved model through REST, but do not know the required Inputs. Is there a documentation for the REST API which is available for the multilingual_universal_encoder?

Many Thanks

Relevant code

saved_model_cli show --dir /home/udot_user_01/serving/saved_models/use-multilingual/3 --all

Relevant log output

I have tried to use the  `saved_model_cli` but I get the following error: 



2022-06-07 11:55:26.082177: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2022-06-07 11:55:27.027213: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-06-07 11:55:27.027290: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['__saved_model_init_op']:
  The given SavedModel SignatureDef contains the following input(s):
  The given SavedModel SignatureDef contains the following output(s):
    outputs['__saved_model_init_op'] tensor_info:
        dtype: DT_INVALID
        shape: unknown_rank
        name: NoOp
  Method name is: 

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['inputs'] tensor_info:
        dtype: DT_STRING
        shape: (-1)
        name: serving_default_inputs:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['outputs'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 512)
        name: StatefulPartitionedCall:0
  Method name is: tensorflow/serving/predict
Traceback (most recent call last):
  File "/home/udot_user_01/miniconda3/lib/python3.9/site-packages/tensorflow/python/framework/ops.py", line 4155, in _get_op_def
    return self._op_def_cache[type]
KeyError: 'SentencepieceOp'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/udot_user_01/miniconda3/lib/python3.9/site-packages/tensorflow/python/saved_model/load.py", line 912, in load_partial
    loader = Loader(object_graph_proto, saved_model_proto, export_dir,
  File "/home/udot_user_01/miniconda3/lib/python3.9/site-packages/tensorflow/python/saved_model/load.py", line 151, in __init__
    function_deserialization.load_function_def_library(
  File "/home/udot_user_01/miniconda3/lib/python3.9/site-packages/tensorflow/python/saved_model/function_deserialization.py", line 409, in load_function_def_library
    func_graph = function_def_lib.function_def_to_graph(
  File "/home/udot_user_01/miniconda3/lib/python3.9/site-packages/tensorflow/python/framework/function_def_to_graph.py", line 82, in function_def_to_graph
    graph_def, nested_to_flat_tensor_name = function_def_to_graph_def(
  File "/home/udot_user_01/miniconda3/lib/python3.9/site-packages/tensorflow/python/framework/function_def_to_graph.py", line 252, in function_def_to_graph_def
    op_def = default_graph._get_op_def(node_def.op)  # pylint: disable=protected-access
  File "/home/udot_user_01/miniconda3/lib/python3.9/site-packages/tensorflow/python/framework/ops.py", line 4159, in _get_op_def
    pywrap_tf_session.TF_GraphGetOpDef(self._c_graph, compat.as_bytes(type),
tensorflow.python.framework.errors_impl.NotFoundError: Op type not registered 'SentencepieceOp' in binary running on 3jw7.l.time4vps.cloud. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/udot_user_01/miniconda3/bin/saved_model_cli", line 8, in <module>
    sys.exit(main())
  File "/home/udot_user_01/miniconda3/lib/python3.9/site-packages/tensorflow/python/tools/saved_model_cli.py", line 1285, in main
    args.func(args)
  File "/home/udot_user_01/miniconda3/lib/python3.9/site-packages/tensorflow/python/tools/saved_model_cli.py", line 747, in show
    _show_all(args.dir)
  File "/home/udot_user_01/miniconda3/lib/python3.9/site-packages/tensorflow/python/tools/saved_model_cli.py", line 314, in _show_all
    _show_defined_functions(saved_model_dir)
  File "/home/udot_user_01/miniconda3/lib/python3.9/site-packages/tensorflow/python/tools/saved_model_cli.py", line 192, in _show_defined_functions
    trackable_object = load.load(saved_model_dir)
  File "/home/udot_user_01/miniconda3/lib/python3.9/site-packages/tensorflow/python/saved_model/load.py", line 782, in load
    result = load_partial(export_dir, None, tags, options)["root"]
  File "/home/udot_user_01/miniconda3/lib/python3.9/site-packages/tensorflow/python/saved_model/load.py", line 915, in load_partial
    raise FileNotFoundError(
FileNotFoundError: Op type not registered 'SentencepieceOp' in binary running on 3jw7.l.time4vps.cloud. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.
 You may be trying to load on a different device from the computational device. Consider setting the `experimental_io_device` option in `tf.saved_model.LoadOptions` to the io_device such as '/job:localhost'.

### tensorflow_hub Version

0.12.0 (latest stable release)

### TensorFlow Version

2.8 (latest stable release)

### Other libraries

_No response_

### Python Version

3.x

### OS

Linux

[gustheman]

I have a non-answer

I can't test the saved_model_cli tool now but the brief results it showed hade the serving_default signature. When you run TFServing, that's the default entry point. So maybe just passing a string will work

There's one specific detail on this model. It needs tensorflow_text for specific ops. To serve it, you can find more information here: https://www.tensorflow.org/tfx/serving/custom_op

hope it helps

[WGierke]

Sorry for the delay.
This apparently requires re-compiling the saved_model_cli tool after one added a dependency on that SentencePieceOp, which is a bit related to google/sentencepiece#325. Doing so will output

signature_def['__saved_model_init_op']:
  The given SavedModel SignatureDef contains the following input(s):
  The given SavedModel SignatureDef contains the following output(s):
    outputs['__saved_model_init_op'] tensor_info:
        dtype: DT_INVALID
        shape: unknown_rank
        name: NoOp
  Method name is: 

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['inputs'] tensor_info:
        dtype: DT_STRING
        shape: (-1)
        name: serving_default_inputs:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['outputs'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 512)
        name: StatefulPartitionedCall:0
  Method name is: tensorflow/serving/predict
The MetaGraph with tag set ['serve'] contains the following ops: {'SentencepieceTokenizeOp', 'RaggedTensorToSparse', 'Placeholder', 'ResourceGather', 'BiasAdd', 'Sqrt', 'Maximum', 'SquaredDifference', 'Identity', 'FloorMod', 'Select', 'Cast', 'Shape', 'SegmentSum', 'Reshape', 'StopGradient', 'Fill', 'ReadVariableOp', 'Pack', 'Relu', 'Rsqrt', 'Range', 'VarHandleOp', 'Unique', 'NoOp', 'MatMul', 'ConcatV2', 'Sum', 'StatefulPartitionedCall', 'Mul', 'SaveV2', 'ParallelDynamicStitch', 'Prod', 'Const', 'Mean', 'RealDiv', 'Tanh', 'PartitionedCall', 'GatherV2', 'NotEqual', 'ShardedFilename', 'Sub', 'Square', 'DynamicPartition', 'AssignVariableOp', 'Size', 'RestoreV2', 'StringJoin', 'MergeV2Checkpoints', 'FloorDiv', 'AddV2', 'Equal', 'SparseToDense', 'StridedSlice', 'ExpandDims', 'SentencepieceOp'}

Concrete Functions:
  Function Name: '__call__'
    Option #1
      Callable with:
        Argument #1
          inputs: TensorSpec(shape=<unknown>, dtype=tf.string, name='inputs')

  Function Name: 'predict'
    Option #1
      Callable with:
        Argument #1
          inputs