nGraph is an open source C++ library, compiler and runtime for Deep Learning frameworks
Switch branches/tags
4630c37d4 adstraw/cpu_quant_debug adstraw/element-type-min-max adstraw/ie-pass-debug adstraw/no-lib-full-path amoitra/memory_manager aprocter/attribute-maps aprocter/comments-for-coordinate-transform aprocter/dynamic-shapes aprocter/exclude-libs-macos aprocter/retire-fancy-ops aprocter/softmax-dynamic-validation aprocter/validate-after-construction arogowiec/lin_quant arogowiec/onnxifi_test arogowiec/test_lin_quant ashoke/dev ashoke_testing avijit/fix-rpath-for-osx awojcik/onnxifi_get_backend_info_required_attributes awojcik/onnxifi_init_backend awojcik/onnxifi_init_graph awojcik/run_graph_add_weights ayzhuang/horizontal_fusion_inception ayzhuang/memory-reuse ayzhuang/wavenet-slice bob/backend_api bob/call_benchmark bob/cgen2 bob/compile_call bob/custom3 bob/docs bob/element_type bob/explicit_compile bob/get_parameters_results bob/gpuh bob/gpuh1 bob/memory_move_op bob/nbench_config bob/ninja bob/python_build bob/python bob/size_rename bob/winbuild cblas_gemm_reshape cconvey/ngraph-shared-mkldnn chrisl/test-general-ci compare_bk cs/bn_provided_vs_batch_statistics cs/bn_update_with_inv_var cs/bn_update2 cs/cmake_take_3 cs/convergence cs/gpu_avgpool_1 cs/gpu_batchnorm_ptx cs/gpu_cudnn_descriptor1 cs/gpu_softmax_cuda2 cs/heterogenous_graph_testing cs/jit_debug cs/maxpoolbprop_update cs/ngraph_comparison_test cs/nvgpu_without_clang cs/nvgpu_without_clang2 cs/op_allocator cs/rnn_old cs/temp cs/v0.10.0-rc.0_ngtfgpu cyphers/bnormbp cyphers/convback cyphers/meta cyphers/tensor cyphers/val dawn/allow-benchmark-tests dawn/external-projects-root dawn/resource-unavailable develop dkannaiy-patch-1 dkannaiy-patch-2 dkannaiy/code-collab-demo dkannaiypatch dkannaiypatch2 dkannaiytest1 doc_more_ops doc_0.11 fboemer/cxx_abi fix-compilation-1444 gauri_conv_add gauri/dcgan_opt gwenger/all_close_f_arg_interpretability gwenger/all_close_f_clarification gwenger/fix_ngraph_test_f_separator harryskim-doc harryskim-patch-1 he igor/osx-codegen intelgpu_backend_any_all_op_implementation jbobba/dynamic-scale-review jmenon/batchnorm jmenon/hacks jmenon/no_codegen_orig jmenon/rn50 korvatkin/rs_refactor krovatkin/PR_MkldnnMemAlocation krovatkin/batchnorm2 krovatkin/bn_fprop_cache_fix krovatkin/bn_optimize krovatkin/bn_split krovatkin/bprop_batchnorm_investigate krovatkin/cacheable_const krovatkin/cblas_gemm2 krovatkin/cblas_gemm3 krovatkin/concat krovatkin/conv_bias krovatkin/cpu_op krovatkin/dex_lk2 krovatkin/find_quantized_region krovatkin/fix_bffloat16 krovatkin/forward_op krovatkin/group_conv_bprop krovatkin/halide3 krovatkin/lstm_recurrent_example krovatkin/mkldnn_aot2 krovatkin/od_diag krovatkin/pass_manager_design krovatkin/pattern_fork_noop krovatkin/pattern_io4 krovatkin/pattern_io5 krovatkin/propagate_inplace_input krovatkin/quantize krovatkin/rebuild krovatkin/remove_reshapes krovatkin/rng krovatkin/shape_matcher krovatkin/stuple krovatkin/trace_node krovatkin/trig krovatkin/used_refactor lam/testing langjian/MPI_bcast langjian/mpi3.0 langjian/test_allreduce louisfeng/NGMX-372_fuse-matmul-test louisfeng/NGRAPH-1097_sum-gemm louisfeng/NGRAPH-1521_lstm-time-fusion louisfeng/debug-module louisfeng/dot_n_2 louisfeng/maskrcnn-layout-fix louisfeng/tvm-dex-only master mbrookhart/fiddle_with_mkldnn_version mlsls nishant_bias_reorder_fix nishant_conv_relu_quantize_PR nishant_dequantize nishant_dynamic_scale nishant_quantize_conv_PR nishant_quantize_conv nishant_quantize_op nishant_quantize_pooling nishant_quantized_concat nishant_sayantan_mnist nivas/onnx_ml nvgpu_tensor_size_bug_fix plaiml pruthvi/MkldnnMemAlocation pruthvi/PR_Bi_Rnn pruthvi/PR_MkldnnMemAlocation pruthvi/bi_rnn pruthvi/bn_backup pruthvi/bn_fprop_cache pruthvi/bn_optimize pruthvi/bn pruthvi/debug pruthvi/experiment pruthvi/fix_bug_lstm_output_replacement pruthvi/fuse_multiple_rnn_layers pruthvi/layer_fuse_rnn pruthvi/lstm_fusion_emitter pruthvi/mkldnn_memory pruthvi/optimize_openMP pruthvi/optimize_rnn_with_TBB pruthvi/rnn_debug pruthvi/variance_pattern_matcher python_binding qdoc_toc r0.8 r0.9 r0.10 r0.11 rancohen/adding-options rearhart/plaidml-perf rmhyman/nbench-fix rmhyman/ngraph_temp_rel_0.9.1 rmhyman/padding_stride_value_change rmhyman/standalone-ngraph-install rob-cmake-ver sandeep/gp sandeep/hb-gpu-cpu-int-3 sandeep/hb-gpu-cpu-int sandeep/hybrid-2 sandeep/hybrid-3 sarkars/adstraw/cpu_quant_debug sharath/disable_bn_folding shared_subgraph_docs silee2/allow_tbb_off silee2/build_macos silee2/cmake_install_rpath silee2/experiment silee2/macos_prebuilt_llvm silee2/manylinux silee2/manylinux1 silee2/mkldnn-v0.12 silee2/openblas silee2/python_builder silee2/rename_wheel silee2/stable_topk silee2/tox testbranch tfl/direct_execution_bug_fix tfl/gpu_dilated_convolution tfl/gpu_one_hot_error_handle tfl/gpu_softmax_optimize tfl/gpu_timer tfl/nvgpu_all_reduce tfl/nvgpu_batchnorm_cuda tfl/nvgpu_broadcast_optimize tfl/nvgpu_hash_map_crash_mpirun tfl/nvgpu_memory_count tfl/nvgpu_optimize_reshape_v2 tfl/nvgpu_reduction_optimization_2 tfl/nvgpu_sum_reduce_cuda tfl/softmax_test tsocha/conv-bias-test tsocha/mainstream-pybind11 tsocha/qlc tvm v0.6 wenzhe/allreduce_debug wnzh/integrate yimei/compare_2_backend_run yixing/cluster-backup yixing/compile-bench yixing/graph-viz yixing/he yixing/nnp-tester yixing/precision yixing/proper-utest yixing/tensor-call
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.
.ci Run ONNX CI on CPU and INTERPRETER (#2084) Nov 21, 2018
cmake add make command to fix-mode (#2229) Dec 14, 2018
contrib/docker Add release notes and improve rendering of (#2117) Nov 27, 2018
doc Integration of MLSL library (#1520) Dec 13, 2018
licenses Add notifications file for eigen (#2147) Nov 30, 2018
maint Use cmake to do style (#2102) Dec 4, 2018
python [ONNX] return single nGraph function with multiple outputs (#2166) Dec 18, 2018
src enable visualize graph even if graphviz is not installed because it j… Dec 18, 2018
test enable visualize graph even if graphviz is not installed because it j… Dec 18, 2018
.clang-format Use weak_ptr for node in inputs/outputs, turn off alignment style. Oct 3, 2017
.gitattributes Normalize line endings (#1649) Sep 20, 2018
.gitignore Windows build support (#2177) Dec 11, 2018
.gitmodules Silee2/single repo (#646) Mar 16, 2018
.travis.yml Travis tests bugfix (#1550) Sep 3, 2018 Harryk remove winml ref (#2204) Dec 10, 2018
CMakeLists.txt add make command to fix-mode (#2229) Dec 14, 2018
CODEOWNERS Update CODEOWNERS for license files and CI/Docker files (#1978) Nov 6, 2018 Architecture and feature docs2 (#2038) Nov 20, 2018 Architecture and feature docs2 (#2038) Nov 20, 2018 Architecture and feature docs2 (#2038) Nov 20, 2018
LICENSE Add LICENSE and switch to Intel Copyright (#466) Feb 8, 2018 Harryk remove winml ref (#2204) Dec 10, 2018 Auto generate version number and apply it to install dir and libngrap… May 3, 2018 Rename runtime::TensorView to runtime::Tensor (#1699) Sep 29, 2018 Harryk remove winml ref (#2204) Dec 10, 2018

nGraph Compiler Stack (Beta)

License Build Status

Quick start

To begin using nGraph with popular frameworks to accelerate deep learning workloads on CPU for inference, please refer to the links below.

Framework (Version) Installation guide Notes
TensorFlow* 1.12 Pip install or Build from source 20 Validated workloads
MXNet* 1.3 Pip install or Build from source 18 Validated workloads
ONNX 1.3 Pip install 14 Validated workloads

Frameworks using nGraph Compiler stack to execute workloads have shown up to 45X performance boost when compared to native framework implementations. We've also seen performance boosts running workloads that are not included on the list of Validated workloads, thanks to our powerful subgraph pattern matching.

Additional work is also being done via PlaidML which will feature running compute for Deep Learning with GPU accleration. See our Architecture and features for what the stack looks like today and watch our Release Notes for recent changes.

What is nGraph Compiler?

nGraph Compiler aims to accelerate developing and deploying AI workloads using any deep learning framework with a variety of hardware targets. We strongly believe in providing freedom, performance, and ease-of-use to AI developers.

The diagram below shows what deep learning frameworks and hardware targets we support. More details on these current and future plans are in the ecosystem section.

nGraph wireframe

While the ecosystem shown above is all functioning, we have validated performance for deep learning inference on CPU processors such as Intel® Xeon®. Please refer to the Release notes to learn more. The Gold release is targeted for April 2019; it will feature broader workload coverage, including quantized graphs, and more detail on our advanced support for int8.

Our documentation has extensive information about how to use nGraph Compiler stack to create an nGraph computational graph, integrate custom frameworks, and to interact with supported backends. If you wish to contribute to the project, please don't hesitate to ask questions in GitHub issues after reviewing our contribution guide below.

How to contribute

We welcome community contributions to nGraph. If you have an idea how to improve it:

  • See the contrib guide for code formatting and style guidelines.
  • Share your proposal via GitHub issues.
  • Ensure you can build the product and run all the examples with your patch.
  • In the case of a larger feature, create a test.
  • Submit a pull request.
  • Make sure your PR passes all CI tests. Note: our Travis-CI service runs only on a CPU backend on Linux. We will run additional tests in other environments.
  • We will review your contribution and, if any additional fixes or modifications are necessary, may provide feedback to guide you. When accepted, your pull request will be merged to the repository.