Sync main branch#1
Merged
EmmaNingMS merged 62 commits intoEmmaNingMS:masterfrom Sep 7, 2019
Merged
Conversation
Add AutoML to 3 main builds. Fix unit tests. Enable copy elision, do not move movable object on return by value.
Implement the first round of changes for quantization inside MLAS. This adds a MatMul operation for U8xU8=S32 for x86/x64 processors.
* update MKLML which has bugfix for thread hang. move PATCH_COMMAND outside BUILD_FOR_NATIVE_MACHINE check. * MKLML_VERSION 2020.0.20190813 is for windows only.
…enums. Relax constraint for enable_all. (#1650)
* Mention OrtCreateSessionFromArray in C API doc * Update perf tool documentation to reflect the new graph optimization enums. Relax constraint for enable_all. * Update one more doc * Update onnx test runner documentation * Add default in the docs
Also cleanup a couple of unused variables.
PyTorch exporter in Pytorch1.2 can natively support multiple opset now
Fix issue that cudnnRNNForwardInferenceEx doesn't support 0 sequence in the bathes Solution: Reset the 0 sequence to 1 for the bathes before call the cudnnRNNForwardInferenceEx, has a array to track the batch id which has 0 sequence. Once get the result, call a CUDA kernel to mask on the output using the batch id tracked in the array.
* Added check for unnecessary function initializations, and removed lock from unneeded areas of code. * Added LRU cache to EP. * Bugfixes for nGraph EP Optimization PR * Changed default cache size to 500 and refactored mutex readability. * Fixed unsafe environmental variable fetch for Windows. * Cleaned up Windows environment functions and cleaned up mutexes.
…g> as input. Fix a bug in the InferenceSession Run() with RunOptions (#1671) - Support bool-Tensor and int8-Tensor in input-output of C# api - Support string-tensor as input in C# api - Fix a bug in InferenceSession.Run() -- RunOptions was not passed into the native call
* update clip for opset 11 * exclude ngraph provider for clip unit tests * exclude ngraph for all clip opset 11 tests * fix op version
* Add support of ReduceSum int64 * add unit test for int64
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
…rdingly. Expose Session/Run log severity levels. (#1615) * Mention OrtCreateSessionFromArray in C API doc * Don't create the default allocator every single time. Rename API accordingly. * Don't create the default allocator every single time. Rename API accordingly. * updates... * updates... * PR comments * fix typo in license header * fix build
Description: make default CPU allocator to use MLAS preferred alignment Motivation and Context This is needed for C API to have an aligned default CPU allocator, the same as the one in CPU provider
- Fix the Windows end-to-end test in NuGet CI - Skip the TestModelSerialization, because it is failing on Linux. Must be fixed before API is released for use. Owner is notified.
* use mlas qgemm for u8u8_s32 gemms * update test
* Updates * Remove preview texts * Update README.md * Updates * Update README.md * Update README.md * Minor wording update * Update README.md * Update doc on CUDA version * revert update * Update readme for issue #1558 * Clean up example section * Cosmetic updates - Add a index of build instructions for browsability - Update build CUDA version from 9.1 to 10 * Fix broken link * Update README to reflect upgrade to pip requirement * Update CuDNN version for Linux Python packages * Clean up content Updated ordering and add table of contents * Minor format fixes * Move Android NNAPI under EP section * Add link to operator support documentation * Fix typo * typo fix * remove todo section
Avoid the need for @PCGOTREL relocations by annotating MLAS global data shared with assembly modules with attribute(visibility("hidden")).
Fix the aarch64 kernel to build properly with the Android NDK (specifically clang).
…ame allocator device (#1715) as long as these providers use the same allocator device Description: Currently ORT throws error when one input is used in different EPs. The change removes that restriction Motivation and Context It is now possible to share inputs across EPs now that allocation are device-based, instead of EP based.
…tom op (#1391) Description: The change adds necessary quantization support on CPU with mixed int8/uint8, as well as int16 for matrix multiply operations that outputs int32 Motivation and Context Integer operations are critical for quantized model's performance Current MatMulInteger implementation in CPU only supports uint8 x uint8, while the spec supports int8 x uint8. Having a default CPU implementation that fully support the spec would help accuracy verification. Besides, some model may need to quantize to int16, but MatMulInteger op does not support that yet. A custom op of MatMulInteger16 is added to satisfy such models.
* Use exec form of ENTRYPOINT for docker server
# Issue
The entrypoint currently uses the shell form - this prevents users from passing in any cmdline arguments... also passing a model_path in means the server only works in the envvar is set... however this is not what the error message says!
```
$ docker run -v /home/rakelkar/try/onnxzoo/style:/mnt/models -it mcr.microsoft.com/onnxruntime/server --model_path /mnt/models/model.onnx
Version: local_build
Commit ID: default
model_path must be the location of a valid file
Allowed options:
-h [ --help ] Shows a help message and exits
--log_level arg (=info) Logging level. Allowed options (case sensitive):
verbose, info, warning, error, fatal
--model_path arg Path to ONNX model
--address arg (=0.0.0.0) The base HTTP address
--http_port arg (=8001) HTTP port to listen to requests
--num_http_threads arg (=4) Number of http threads
--grpc_port arg (=50051) GRPC port to listen to requests
```
# Fix
1. remove the env var
2. use the exec form
* Update readme to use model_path arg
…1679) * Support bilinear mode with actual 2D inputs in Resize and upsample * Fix build break * Fix build break * Add test * CUDA changes * Resolve PR comments * Resolve comments
…in 0.5 release. (#1694) * Mention OrtCreateSessionFromArray in C API doc * Fix registration of Equal op causing one of the automl models to break in 0.5 release. * updates...
…which cause huge data copy. If the node's inputs are all initializer, we shouldn't fallback the node to CPU. (#1727) Fix an issue that CUDA EP fallback too much nodes to CPU for some case which cause huge data copy. #1675 Currently, if the node's inputs are all as initialier, CUDA EP will fallback it to CPU. And it will also fallback some nodes under it. It could cause some huge data copy. for the case reported by a user, it has several Slices with input from initializer, and a Concat op to concat the output from Slice output. The data is huge 16MB after concat, which make the data copy from CPU to GPU quite costly because it's a sync copy. Fix If the node's inputs are all initializer, we shouldn't fallback the node to CPU.
Update the docker file for OpenVINO which is used for AML
Fix typo in NMS code
* moved subgraph_index to MklDnn Execution Provider * code cleanup
* Implement Nuphar execution provider Nuphar execution provider is a TVM-based compilation provider. It has shown great speedups for RNN models using Scan. This PR is mainly for a preview of the shared codegen library for other TVM-based providers. * Fix submodules * Fix TVM submodule * Update Nuphar to latest and resolve confliction * Remove stale files caused by merge -X theirs * Revert heap buffer change to not introduce onnxruntime_framework into onnxruntime_perf_test * Fix bad merge * Merge from Nuphar * Fix warning treated as error, revert some unnecessary changes * Revert some more test changes * Some more test revert or comments to make review easier New tests could be added later * One more revert of unnecessary changes * More change revert. Test could be added back later.
* Mention OrtCreateSessionFromArray in C API doc * Enforce shape validation. * Update broken models
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
Restore the missing variable
* Update README.md * Update onnx-inference-byoc-gpu-cpu-aks.ipynb * Update README.md
…ions (unless where not possible). (#1730) * Mention OrtCreateSessionFromArray in C API doc * Fix perf test executable due to removal of certain C APIs * fix linux build * Avoid duplication * Update coding guidelines to prefer using make_unique for heap allocations (unless where not possible).
When NUPHAR_USE_MKL or NUPHAR_USE_AVX2 is not defined, we got "unreachable code" warnings on Windows, which were truned into errors and broke the build.
Enable Nuphar EP docker build Revert back to LLVM 6.0.1 Reinstate disabled Softmax tests caused by LLVM 8.0.1 Reinstate Nuphar Python test due to stale sympy version Increase build timeout of Linux CI
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description: Describe your changes.
Motivation and Context