[WIP] Integrating OpenVINO Runtime in Spark NLP #13947
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reopened as #14200
This PR introduces OpenVINO Runtime support in Spark NLP
Description
This PR enables Spark NLP to leverage the OpenVINO Runtime API for Java to load and run models in various formats including ONNX, PaddlePaddle, Tensorflow, Tensorflow Lite and OpenVINO IR format. OpenVINO also enables performance improvements when running on supported intel hardware, with upto 40% improvement vs Tensorflow on benchmarks with no further tuning. You can also take advantage of the full optimization and quantization capabilities offered by the OpenVINO toolkit when exporting/converting the model to the OpenVINO format using the Model Conversion API.
The following annotators have been enabled to work with OpenVINO:
Note: To take advantage of this feature, see these instructions to build OpenVINO jar (Linux), and these to build Spark NLP. OpenVINO is cross-platform. Refer here for Windows build instructions, and here for other platforms.
Motivation and Context
Screenshots (if appropriate):
Types of changes
Checklist: