Serve a different language model?

[johann-petrak]

I followed the instructions from here:
https://github.com/dsindex/syntaxnet/blob/master/README_api.md

This allowed me to run the server and use it using a client based on the parsey_api.proto and sentence.proto files (I implemented my own Java client for this).

What I do not understand is how to best switch the model to one of "Parsey's cousins" ? Eg if I download the German model http://download.tensorflow.org/models/parsey_universal/German.zip, how can I use that model instead of the English one? Can I use the same client with that model?

[dsindex]

# copy parsey_mcparseface.py to serving/tensorflow_serving/example
# buid it
$ cd ../
$ cp api/parsey_mcparseface.py serving/tensorflow_serving/example
$ cd serving
$ bazel --output_user_root=bazel_root build --nocheck_visibility -c opt -s //tensorflow_serving/example:parsey_mcparseface --genrule_strategy=standalone --spawn_strategy=standalone --verbose_failures
$ ls bazel-bin/tensorflow_serving/example/parsey_mcparseface

# run
# this will read model from --model_dir and export to --export_path directory
$ bazel-bin/tensorflow_serving/example/parsey_mcparseface --model_dir=syntaxnet/models/parsey_mcparseface --export_path=exported

i think you can export parsey_mcparseface model using above commands.
so, if you download the German model and copy the contents to the parsey directory,
then you can also export the German model.
but i did not test this trick ;;

it might be needed to modify context.pbtxt.

[johann-petrak]

I actually tried this by running all these commands up to the last one.

Then I extracted the German.zip model in the root directory (so a German directory was created in parallel to English)

Then from ./serving I ran:

bazel-bin/tensorflow_serving/example/parsey_mcparseface --model_dir=../German --export_path=exported_German

This produced the following error:

W external/org_tensorflow/tensorflow/core/framework/op_kernel.cc:955] Not found: ../German/context.pbtxt
E external/org_tensorflow/tensorflow/core/common_runtime/executor.cc:334] Executor failed to create kernel. Not found: ../German/context.pbtxt
         [[Node: fs_brain_tagger/FeatureSize = FeatureSize[arg_prefix="brain_tagger", task_context="../German/context.pbtxt", _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
W external/org_tensorflow/tensorflow/core/framework/op_kernel.cc:955] Not found: ../German/context.pbtxt
E external/org_tensorflow/tensorflow/core/common_runtime/executor.cc:334] Executor failed to create kernel. Not found: ../German/context.pbtxt

Since I do not really understand how the models and the way how tensorflow serving runs them between languages I gave up assuming I was doing something stupid :)

So in the case of German, the context.pbtxt file does not even exist in the model directory.
This is the list of all files in the model directory:

char-map        checkpoint  morpher-params   morphology-map  prefix-table  tagger-params  tokenizer-params       word-map
char-ngram-map  label-map   morph-label-set  parser-params   suffix-table  tag-map        tokenizer-params.meta

I also had already tried just brutally replacing the files in serving/syntaxnet/models/parsey_mcparseface/ with the files from the German model but that failed with the same error about the missing context.pbtxt file. When I just copied over the missing files from the English model I got some error about a dimensionality mismatch.

TBH, I really do not understand much of this, I was just hoping that once I have installed SyntaxNet and get it to run with tensorflow serving, which I could do thanks to your instructions, I could easily get it to work with all other languages by just replacing the model files, but apparently that does not work.

[dsindex]

@johann-petrak

i investigated that problems.
unfortunately,
there is a version mismatch between the tf_models which generated parsey's cousin and tf_models( a4b7bb9a5dd2c021edcd3d68d326255c734d0ef0 ) you checked out.

...
$ git clone https://github.com/dsindex/syntaxnet.git work
$ cd work
$ git clone --recurse-submodules https://github.com/tensorflow/serving
# checkout proper version of tf_models
$ cd serving/tf_models
$ git checkout a4b7bb9a5dd2c021edcd3d68d326255c734d0ef0
...

one solution might be patching serving/tf_models up to date.
but, these patch was originally developed by @dmansfield.
( dmansfield/parsey-mcparseface-api#1 )

another solution is ....

what about training the German model directly by using the Universal Treebank Corpus?

https://github.com/dsindex/syntaxnet#training-tagger-and-parser-with-another-corpus

in this case, you have to checkout the version 'a4b7bb9a5dd2c021edcd3d68d326255c734d0ef0'
and then follow instructions :

1. https://github.com/dsindex/syntaxnet
2. https://github.com/dsindex/syntaxnet/blob/master/README_api.md

git clone --recursive https://github.com/tensorflow/models.git
cd models
git checkout a4b7bb9a5dd2c021edcd3d68d326255c734d0ef0
cd models/syntaxnet/tensorflow
./configure
cd ..
bazel test syntaxnet/... util/utf8/...

[johann-petrak]

Thank you very much for looking into this!
I would very much prefer to find some way to deploy the downloadable models somehow rather than re-training all those models on my own (I only used German as an example).
Did you find out which version of tf_models was used for parsey's cousins? I am not sure I understand what you say about patching: are you referring to pr250-patch-a4b7bb9a.diff.txt?

[dsindex]

@johann-petrak

yes, i refered that patch(pr250-patch-a4b7bb9a.diff.txt).

https://github.com/dsindex/syntaxnet#parseys-cousins
few days ago, i checked out models up to date.
( this version tensorflow/models@a5d45f2 )
and then downloaded those parsey's cousins and tested.
( https://github.com/tensorflow/models/blob/master/syntaxnet/universal.md )
it just worked fine.
so, i guess that the parsey's counsins had been trained by around that version.

[znavoyan]

Hello, I have the same problem. I'm trying to setup Spanish model to work with the Serving. I patched as described, with pr250-patch-a4b7bb9a.diff.txt, downloaded Spanish model. For the rest I followed
https://github.com/dsindex/syntaxnet/blob/master/README_api.md link "export model" section.
However while executing

bazel-bin/tensorflow_serving/example/parsey_mcparseface --model_dir=syntaxnet/models/Spanish --export_path=exported_spanish,

I get the same error as described below:

W external/org_tensorflow/tensorflow/core/framework/op_kernel.cc:955] Not found: syntaxnet/models/Spanish/context.pbtxt
E external/org_tensorflow/tensorflow/core/common_runtime/executor.cc:334] Executor failed to create kernel. Not found: syntaxnet/models/Spanish/context.pbtxt
         [[Node: fs_brain_tagger/FeatureSize = FeatureSize[arg_prefix="brain_tagger", task_context="syntaxnet/models/Spanish/context.pbtxt", _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
W external/org_tensorflow/tensorflow/core/framework/op_kernel.cc:955] Not found: syntaxnet/models/Spanish/context.pbtxt
E external/org_tensorflow/tensorflow/core/common_runtime/executor.cc:334] Executor failed to create kernel. Not found: syntaxnet/models/Spanish/context.pbtxt

Can you please point out what I have missed?

[dsindex]

@znavoyan
as i mentioned above comment, a model from parsey's cousin is not able to export by that patch due to version mismatch.
so, i recommend you to train the spanish model via the universial dependency corpus and then export.

[szz-dvl]

Hi;

I've been trying to reproduce the steps mentioned above by johann-petrak, it is, trying to export a pre-trained model (Spanish-AnCora particularly) without success, getting the same error mentioned above. I used "context.pbtxt.template" to generate a context.pbtxt file pointing to the model files without success because "tag-to-category" file was missing. After this I downloaded the Treebanks Spanish-AnCora corpus and trained the model (using the script found at https://github.com/dsindex/syntaxnet.git) getting the needed files to export it, however this time I'm missing the file "tagger-params". Actually the above mentioned script yields a directory called "tagger-params" with a file "model.meta" inside of it, I tried to use this file as "tagger-params" unsuccessfully, the error I'm getting is the following:

Could not open ../corpus/models/UD_Spanish-AnCora/tagger-params: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?

So I'm stuck at this point and any help will be really appreciated.

Thanks in advance.

Szz.

[dsindex]

@szz-dvl

i guess ... you need to modify 'train.sh' to use
xcopy_model instead copy_model.

[szz-dvl]

@dsindex:

I copied the required files manually, now I've my "tagger-params", "tagger-params.meta", "parser-params" and "parser-params.meta" files in the model directory, however when I try to export the model I get:

Traceback (most recent call last): File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/tensorflow_serving/example/parsey_mcparseface.py", line 188, in <module> tf.app.run() File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/external/org_tensorflow/tensorflow/python/platform/app.py", line 30, in run sys.exit(main(sys.argv)) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/tensorflow_serving/example/parsey_mcparseface.py", line 172, in main model[prefix]["documents"] = Build(sess, source, model[prefix]) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/tensorflow_serving/example/parsey_mcparseface.py", line 79, in Build parser.saver.restore(sess, model_path) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/external/org_tensorflow/tensorflow/python/training/saver.py", line 1129, in restore {self.saver_def.filename_tensor_name: save_path}) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/external/org_tensorflow/tensorflow/python/client/session.py", line 710, in run run_metadata_ptr) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/external/org_tensorflow/tensorflow/python/client/session.py", line 908, in _run feed_dict_string, options, run_metadata) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/external/org_tensorflow/tensorflow/python/client/session.py", line 958, in _do_run target_list, options, run_metadata) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/external/org_tensorflow/tensorflow/python/client/session.py", line 978, in _do_call raise type(e)(node_def, op, message)

tensorflow.python.framework.errors.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [6045,16] rhs shape= [5,4]
[[Node: save/Assign_2 = Assign[T=DT_FLOAT, _class=["loc:@brain_tagger/embedding_matrix_1"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/cpu:0"](brain_tagger/params/brain_tagger/embedding_matrix_1/ExponentialMovingAverage, save/restore_slice_2)]]

Caused by op u'save/Assign_2', defined at: File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/tensorflow_serving/example/parsey_mcparseface.py", line 188, in <module> tf.app.run() File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/external/org_tensorflow/tensorflow/python/platform/app.py", line 30, in run sys.exit(main(sys.argv)) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/tensorflow_serving/example/parsey_mcparseface.py", line 172, in main model[prefix]["documents"] = Build(sess, source, model[prefix]) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/tensorflow_serving/example/parsey_mcparseface.py", line 77, in Build parser.AddSaver(slim_model) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/external/syntaxnet/syntaxnet/graph_builder.py", line 568, in AddSaver self.saver = tf.train.Saver(variables_to_save) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/external/org_tensorflow/tensorflow/python/training/saver.py", line 861, in __init__ restore_sequentially=restore_sequentially) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/external/org_tensorflow/tensorflow/python/training/saver.py", line 519, in build filename_tensor, vars_to_save, restore_sequentially, reshape) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/external/org_tensorflow/tensorflow/python/training/saver.py", line 282, in _AddRestoreOps validate_shape=validate_shape)) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/external/org_tensorflow/tensorflow/python/ops/gen_state_ops.py", line 45, in assign use_locking=use_locking, name=name) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/external/org_tensorflow/tensorflow/python/framework/op_def_library.py", line 703, in apply_op op_def=op_def) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/external/org_tensorflow/tensorflow/python/framework/ops.py", line 2317, in create_op original_op=self._default_original_op, op_def=op_def) File "/path/to/SyntaxNet/work/serving/bazel-bin/tensorflow_serving/example/parsey_mcparseface.runfiles/tf_serving/external/org_tensorflow/tensorflow/python/framework/ops.py", line 1239, in __init__ self._traceback = _extract_stack()

I will try to retrain the model now, however if you know the cause of the error please make me know. I will update the post if I come to any success.

Thanks in advance.

Szz.

[dsindex]

@szz-dvl

i thought you followed this instructions(https://github.com/dsindex/syntaxnet/blob/master/README_api.md)

then, this issue may help you.
#19

exporting a syntaxnet model is very tricky and you should use the designated version of syntaxnet.

1. install syntaxnet(designated version)

$ git clone --recurse-submodules https://github.com/tensorflow/serving
# checkout proper version of serving
$ cd serving
$ git checkout 89e9dfbea055027bc31878ee8da66b54a701a746
$ git submodule update --init --recursive
# checkout proper version of tf_models
$ cd tf_models
$ git checkout a4b7bb9a5dd2c021edcd3d68d326255c734d0ef0

2. training

# after installing syntaxnet.
# gpu supporting : https://github.com/tensorflow/models/issues/248#issuecomment-288991859
$ pwd
/path/to/tf_models/syntaxnet
$ git clone https://github.com/dsindex/syntaxnet.git work
$ cd work
$ echo "hello syntaxnet" | ./demo.sh
# training parser only with parsed corpus
$ ./parser_trainer_test.sh

....

as you see, these steps are horrible ;; so, i recommend you to use DRAGNN.
( you can use up-to-date version of syntaxnet in this case )
https://github.com/dsindex/syntaxnet/blob/master/README.md#dragnn

[szz-dvl]

Hey @dsindex!

Thanks for the quick reply, I followed the instructions you mention above (checked out proper versions for everything and applied the mentioned patch too) with no success as you can see ... I took a look at the issue you point too, however I'm not sure it is valid for my case. I'm now retraining the model, I will try to export it again after the process finishes. If this doesn't work I will try to rebuild syntaxnet and train the model directly as you point here ... Thanks for the DRAGNN suggestion too, however I will try to insist a little bit more with this, it fit more my needs. I will update the post if I get something!

Thanks again!

Szz.

[vishu0047]

Hello @dsindex

I am trying to reproduce the step mentioned on "https://github.com/dsindex/syntaxnet/blob/master/README_api.md" , trying to export my pre-trained English model but i am getting below error on running the step:
bazel --output_user_root=bazel_root build --nocheck_visibility -c opt -s //tensorflow_serving/example:parsey_api

error is:
Extracting Bazel installation...
........
ERROR: Failed to load Skylark extension '//tensorflow/tensorflow:workspace.bzl'.
It usually happens when the repository is not defined prior to being used.
Maybe repository '' was defined later in your WORKSPACE file?
ERROR: cycles detected during target parsing.
INFO: Elapsed time: 2.298s
./batch.sh: line 52: ./bazel-bin/tensorflow_serving/example/parsey_api: No such file or directory

So I'm stuck at this point and any help will be really appreciated.

Thanks in advance.

[dsindex]

@vishu0047

the error you encountered looks very similar to the issue ( #17 ).

did you install the bazel version 0.2.2b ?

[aluminumbox]

Hi @dsindex . Sorry to bother you again.
Thanks to your help, now I have my own script to train and parse sentence. My next step is to deploy the model as a service, however, as you know, the parse_eval.py needs to reinitialize the graph after each request, which takes too much time.
As you mentioned above:

as you see, these steps are horrible ;; so, i recommend you to use DRAGNN.
( you can use up-to-date version of syntaxnet in this case )
https://github.com/dsindex/syntaxnet/blob/master/README.md#dragnn

I know that one solution is to export the syntaxnet model and run it using tf_serving, which as you said is tricky and horrible. So, you recommended dragnn and in your repository, there is a dragnn.sh. Can it work like wait for user's input and output the result, and then wait for response withouth reinitializing the graph? I know I should try it before asking questions, but the build process really takes a lot of time. It will be a lot of help if you can point out the direction for me.

[dsindex]

yes!
dragnn is much better solution for serving api. if you look at inference_dragnn.py, you can see :

model = load_model()
while 1 :
... do-inference(model)
unload_model(model)

i guess this style is what you want.

in ‘web api using tornado’ section in README, i described how to build an tornado api.
but, session of tensorflow is not fork-safe. therefore, you should modify the api code. for example, create session after forking.

[aluminumbox]

@dsindex Yes, that is exactly what I want to achieve, and thanks for the remind.

[aluminumbox]

hi @dsindex .I was reading your code. It seems that you used dragnn and trained your own model based on the thoery behind syntaxnet. so, there is no context.pbtxt now, and the model is easy to be exported and deploy.
One curiosity, in syntaxnet, during the training, it first trained pos tagger, then local paser training and global parser training. During the evaluation, it went through morpher, pos tagger and parser. Did your dragnn train and test through the same steps, or you provided a base model and we shoud modify to accomplish the training and evaluate steps in syntaxnet?

[dsindex]

um...
train and inference are using same master_spec which defines look-ahead, tagger, parser.
you can see that build_train_graph, build_inference_graph have the same parameter, already built ‘master_spec’.

[aluminumbox]

Hi @dsindex
Thanks for your help, I successfully got your dragnn model running. My syntaxnet version is:
bc0edaf8ec635c2a493a9303071e3d2fe97f3b7b. So, I guess your train_dragnn.sh/test_dragnn.sh works fine with other syntaxnet version.
I see that you defined lookahead/tagger/parser in model_dragnn.build_master_spec, created parser_spec.textproto, and model_dragnn.load_master_spec used parser_spec_textproto and resource files to generate 'master_spec'.
So, in your dragnn model, the parameters are defined in 'master_spec'. In syntaxnet, the parameters are defined in context.pbtxt. What is their relation? How to get the model with the same setting as the one in syntaxnet parsey_universal? For example, I want to add a morpher analysis to the parse result, but I don't know how to edit the model_dragnn.build_master_spec module.

[dsindex]

well~
i made ‘master_spec’ by following the dragnn tutorial. so, i didn’t try to fit exactly to context.pbtxt in syntaxnet.
because those models are different, i thought that ‘do i need to use same features?’, ‘hmm.. that is enough at this time’

and for morpher, did you mean segmentator? i didn’t find a segmentator using dragnn system.
but in source code, ‘morpher’ is available value like ‘tagger’. so, i just guess it would be possible.

for korean language, it is hard to accomplish morphological analyzer by using syntaxnet. so, i used a corpus which already has morphological analyzed results.

[aluminumbox]

@dsindex You may notice that in parse_universal/parse.sh, there are three different arg_prefix: brain_morphre/brain_tagger/brain_parser. The morpher gives something like this Mood=Ind|Tense=Pres|VerbForm=Fin|fPOS=AUX++VBP, which is the 6th element CoNLL-U Format.
In dragnn, there is a graph_builder.MasterBuilder that returns a builder for a DRAGNN stack of models.
class MasterBuilder(object): """A builder for a DRAGNN stack of models.
In syntaxnet, there is also a graph_builder.greedyparser that builds a Chen & Manning style greedy neural net parser.
class GreedyParser(object): """Builds a Chen & Manning style greedy neural net parser.
As the repository contains two parts: dragnn and syntaxnet, I believe that syntaxnet should used dragnn during the whole process, so context.pbtxt should transform into master_spec somehow. I haven't found their connection yet. Please let me know if you have any clue, or is it that they are just irrelevant? Many thanks!