Add basic grpc MT server #1807

ryanleary · 2021-02-26T05:14:43Z

Add readme, server updates

Signed-off-by: Ryan Leary rleary@nvidia.com

Add readme, server updates Signed-off-by: Ryan Leary <rleary@nvidia.com>

lgtm-com · 2021-02-26T05:24:15Z

This pull request introduces 5 alerts when merging 619df56 into efaab9e - view on LGTM.com

new alerts:

5 for Unused import

MaximumEntropy · 2021-02-26T19:16:10Z

examples/nlp/machine_translation/nmt_server/server.py

+        request_strings = [x for x in request.texts]
+
+        for batch in batches(request_strings, self._batch_size):
+            batch_results = self._models[lang_pair].translate(text=batch, source_lang=None, target_lang=None)


Is there a reason why source and target lang is None here? This won't use the correct input tokenization and target detokenization.

Nope, no good reason. Better question is why doesn't the model already know the source_lang and target_lang if it matters? It's being restored from a nemo model.. that data should already be there.

We've added that now and new models won't require this (#1805). But for existing models, and for temporary backward compatibility, the translate method has a source_lang and target_lang that need to be set.

lgtm-com · 2021-02-26T23:28:17Z

This pull request introduces 5 alerts when merging f044219 into 4d6cad6 - view on LGTM.com

new alerts:

5 for Unused import

lgtm-com · 2021-03-04T05:40:05Z

This pull request introduces 5 alerts when merging 79d9ee7 into 91774a7 - view on LGTM.com

new alerts:

5 for Unused import

lgtm-com · 2021-03-05T23:57:02Z

This pull request introduces 5 alerts when merging 645d84e into 7ecf04f - view on LGTM.com

new alerts:

5 for Unused import

lgtm-com · 2021-03-08T19:02:19Z

This pull request introduces 5 alerts when merging 4064008 into 52c20a1 - view on LGTM.com

new alerts:

5 for Unused import

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

tools/nmt_grpc_service/api/nmt_pb2.py

tools/nmt_grpc_service/api/nmt_pb2_grpc.py

tools/nmt_grpc_service/nmt.proto

ericharper · 2021-08-06T22:48:41Z

tools/nmt_grpc_service/server.py

+        if not hasattr(model, "src_language") or not hasattr(model, "tgt_language"):
+            raise ValueError(f"Could not find src_language and tgt_language in model attributes")
+


Can we add these to the argparser? Then if a model doesn't have them, it could be updated by the user specified languages?

ericharper

LGTM! Thanks!

Only some minor comments.

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

lgtm-com · 2021-08-09T20:26:12Z

This pull request introduces 6 alerts when merging 6b474ed into 241855c - view on LGTM.com

new alerts:

4 for Unused import
2 for Modification of parameter with default

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

lgtm-com · 2021-08-09T20:41:41Z

This pull request introduces 2 alerts when merging 323d938 into 241855c - view on LGTM.com

new alerts:

2 for Modification of parameter with default

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

lgtm-com · 2021-08-09T21:24:26Z

This pull request introduces 2 alerts when merging af51a3d into 241855c - view on LGTM.com

new alerts:

2 for Modification of parameter with default

lgtm-com · 2021-08-09T22:08:30Z

This pull request introduces 2 alerts when merging e3f7898 into 7e6197d - view on LGTM.com

new alerts:

2 for Modification of parameter with default

* Add basic grpc MT server Add readme, server updates Signed-off-by: Ryan Leary <rleary@nvidia.com> * style fix Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> * fixing license headers Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> * Add punctuation model into NMT service Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix merge conflicts Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * style fixes to unblock CI Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> * Add a Jarvis ASR + NeMo NMT client Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * style fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Refactor gRPC service Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update license headers Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update one more license header Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Whitepsace in header Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix grpc requirement Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update license headers Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add option to specify src/tgt lang and import fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix unused imports Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Renaming variables Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Ryan Leary <rleary@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Co-authored-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add basic grpc MT server Add readme, server updates Signed-off-by: Ryan Leary <rleary@nvidia.com> * style fix Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> * fixing license headers Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> * Add punctuation model into NMT service Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix merge conflicts Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * style fixes to unblock CI Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> * Add a Jarvis ASR + NeMo NMT client Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * style fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Refactor gRPC service Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update license headers Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update one more license header Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Whitepsace in header Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix grpc requirement Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update license headers Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add option to specify src/tgt lang and import fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix unused imports Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Renaming variables Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Ryan Leary <rleary@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Co-authored-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Paarth Neekhara <paarth.n@gmail.com>

* Add basic grpc MT server Add readme, server updates Signed-off-by: Ryan Leary <rleary@nvidia.com> * style fix Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> * fixing license headers Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> * Add punctuation model into NMT service Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix merge conflicts Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * style fixes to unblock CI Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> * Add a Jarvis ASR + NeMo NMT client Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * style fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Refactor gRPC service Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update license headers Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update one more license header Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Whitepsace in header Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix grpc requirement Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update license headers Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add option to specify src/tgt lang and import fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix unused imports Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Renaming variables Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Ryan Leary <rleary@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Co-authored-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Add basic grpc MT server

619df56

Add readme, server updates Signed-off-by: Ryan Leary <rleary@nvidia.com>

MaximumEntropy reviewed Feb 26, 2021

View reviewed changes

Merge branch 'main' into nmt_grpc

f044219

Merge branch 'main' into nmt_grpc

79d9ee7

Merge branch 'main' into nmt_grpc

645d84e

Merge branch 'main' into nmt_grpc

4064008

okuchaiev and others added 19 commits March 8, 2021 16:59

Merge branch 'main' into nmt_grpc

991f111

style fix

742fa69

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

fixing license headers

a996956

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

Add punctuation model into NMT service

bd96e32

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Merge branch 'main' into nmt_grpc

34a1bb1

Merge branch 'nmt_grpc' of https://github.com/ryanleary/NeMo into HEAD

8fdaa36

Merge branch 'main' into nmt_grpc

472803b

Fix merge conflicts

f9f3593

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Merge branch 'main' into nmt_grpc

f595535

style fixes to unblock CI

14f6505

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

Merge branch 'main' of https://github.com/NVIDIA/NeMo into HEAD

01e4fd4

Add a Jarvis ASR + NeMo NMT client

c92356a

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

style fixes

3b697ae

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Merge branch 'nmt_grpc' of https://github.com/ryanleary/NeMo into HEAD

9e19a6c

Merge branch 'main' into nmt_grpc

878fc99

Merge branch 'main' into nmt_grpc

ce00cbf

Merge branch 'main' into nmt_grpc

8f63b92

Merge branch 'main' into nmt_grpc

d99adaf

Merge branch 'main' into nmt_grpc

30fae87

MaximumEntropy added 8 commits August 6, 2021 13:37

Refactor gRPC service

9d0b4d2

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Merge branch 'nmt_grpc' of https://github.com/ryanleary/NeMo into HEAD

d4d9795

Merge branch 'main' into nmt_grpc

de3cc48

Update license headers

f9e1fbe

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Update one more license header

7357a65

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Whitepsace in header

34b92a6

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Style fixes

b5ebecc

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Fix grpc requirement

b2eb5ba

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

MaximumEntropy requested a review from ericharper August 6, 2021 22:05

ericharper reviewed Aug 6, 2021

View reviewed changes

tools/nmt_grpc_service/api/nmt_pb2.py Outdated Show resolved Hide resolved

ericharper reviewed Aug 6, 2021

View reviewed changes

tools/nmt_grpc_service/api/nmt_pb2_grpc.py Outdated Show resolved Hide resolved

ericharper reviewed Aug 6, 2021

View reviewed changes

tools/nmt_grpc_service/nmt.proto Outdated Show resolved Hide resolved

ericharper reviewed Aug 6, 2021

View reviewed changes

ericharper approved these changes Aug 6, 2021

View reviewed changes

MaximumEntropy added 5 commits August 9, 2021 09:26

Merge branch 'main' into nmt_grpc

306ff84

Update license headers

edb60b9

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Merge branch 'nmt_grpc' of https://github.com/ryanleary/NeMo into HEAD

a41f21f

Add option to specify src/tgt lang and import fixes

3399cc7

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Style fixes

6b474ed

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Fix unused imports

323d938

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Renaming variables

af51a3d

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Merge branch 'main' into nmt_grpc

e3f7898

ericharper merged commit 2be5853 into NVIDIA:main Aug 9, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add basic grpc MT server #1807

Add basic grpc MT server #1807

ryanleary commented Feb 26, 2021

lgtm-com bot commented Feb 26, 2021

MaximumEntropy Feb 26, 2021

ryanleary Feb 26, 2021

MaximumEntropy Feb 26, 2021

lgtm-com bot commented Feb 26, 2021

lgtm-com bot commented Mar 4, 2021

lgtm-com bot commented Mar 5, 2021

lgtm-com bot commented Mar 8, 2021

ericharper Aug 6, 2021

ericharper left a comment

lgtm-com bot commented Aug 9, 2021

lgtm-com bot commented Aug 9, 2021

lgtm-com bot commented Aug 9, 2021

lgtm-com bot commented Aug 9, 2021

		if not hasattr(model, "src_language") or not hasattr(model, "tgt_language"):
		raise ValueError(f"Could not find src_language and tgt_language in model attributes")

Add basic grpc MT server #1807

Add basic grpc MT server #1807

Conversation

ryanleary commented Feb 26, 2021

lgtm-com bot commented Feb 26, 2021

MaximumEntropy Feb 26, 2021

Choose a reason for hiding this comment

ryanleary Feb 26, 2021

Choose a reason for hiding this comment

MaximumEntropy Feb 26, 2021

Choose a reason for hiding this comment

lgtm-com bot commented Feb 26, 2021

lgtm-com bot commented Mar 4, 2021

lgtm-com bot commented Mar 5, 2021

lgtm-com bot commented Mar 8, 2021

ericharper Aug 6, 2021

Choose a reason for hiding this comment

ericharper left a comment

Choose a reason for hiding this comment

lgtm-com bot commented Aug 9, 2021

lgtm-com bot commented Aug 9, 2021

lgtm-com bot commented Aug 9, 2021

lgtm-com bot commented Aug 9, 2021