Generic model builder and inference engine

docs/XModelRunner.md

@@ -0,0 +1,44 @@
+## XModelRunner
+
+Glow supplies a number of out-of-the-box model builders/runners, which include
+the `ImageClassifier` and the `ModelLoader`. The former is tailored towards specific 
+networks built for image classification, while the latter is a generic model loader (see
+the corresponding documentation). 
+
+`XModelRunner` is a generic model builder/runner that is able to consume any model --
+either in the ONNX format, or as a Caffe2 protobuf -- and
+either compile it (producing a bundle if requested, along with profiling when requested), 
+or run inference on it. `XModelRunner`
+is built along with the other models using the standard build instructions (see the 
+building documentation). Below is a description of the command line options for `XModelRunner`.
+
+### Command Line Options
+
+In addition to all of the "standard" command line options (i.e. those that are common to
+all model builders/runners, and can be obtained with the `-help` option), a few
+options are supported by `XModelRunner`, some of which are mandatory and some are optional.
+
+#### Required Options
+
+| Option | Expected Values | Description | Required |
+| :------ | :------ | :------ | :------ |
+|`input-tensor-dims` | Comma-separated list of ints | Input tensor dimensions | Yes |
+|`output-tensor-names` | Comma-separated list of strings (no spaces) | Output tensor names (can be more than one) | Yes |
+|`write-output` | Boolean flag | Whether to write output to output files (only applicable when not building a bundle) | No (default is `False`) |
+
+#### Expected Input
+
+`XModelRunner` expects either a list of file names containing input (one input tensor per file), which are positional arguments, or a file name containing the list of file names that contain input (one file name per line), specified with the `input-file-list` option. Input specification is required when either profiling the network, or running inference (i.e. not building a bundle). Otherwise, input can be omitted.
+
+#### Produced Output
+
+When input files are specified, output is produced (saved into binary files) only when the `write-output` option is specified. In this case, the input file name acts as a base name for the output file name. The output file name is composed as `[input file name].out.dat`. When input is not specified and bundles are requested, the runner produces bundles in the specified output directory. Also, `write-output` may be omitted when profiling the network. 
+
+### Technical Notes
+
+1. The runner currently supports only one (named) input tensor, but can work with multiple (named) output tensors.
+2. The runner does not currently support streaming input, but this feature is planned.
+
+### A Note on the Initial Contribution
+
+The initial contribution of this runner (as well as the corresponding documentation) was made as part of the open source contribution initiative by the XPERI Corporation (xperi.com). 

inference_engines/x-inference-engines/CMakeLists.txt

@@ -0,0 +1,66 @@
+cmake_minimum_required(VERSION 3.0)
+project(x-infer)
+
+option(LIB_ONLY "Only build the library" OFF)
+option(BUILD_FOR_DYNAMIC_LINKAGE "Whether to build for dynamic or static bundle linkage" ON)
+option(LINK_LIBS_STATICALLY "Whether to link in all the required libraries statically" ON)
+option(ENABLE_PERF_MONITORING "Whether to enable performance monitoring" OFF)
+
+set(EXECUTABLE_OUTPUT_PATH ${CMAKE_BINARY_DIR}/bin)
+set(LIBRARY_OUTPUT_PATH ${CMAKE_BINARY_DIR}/bin)
+
+if(LINK_LIBS_STATICALLY)
+    message(WARNING "Static linking may cause issues with dlopen on target")
+    message("Will statically link executable")
+    set(TARGET_COMPILE_FLAGS "-static"
+    )
+    set(TARGET_LINKER_FLAGS "-static"
+    )
+else()
+    set(TARGET_COMPILE_FLAGS )
+endif()
+
+if (DEFINED TARGET_OPTIONS)
+    set(TARGET_COMPILE_FLAGS "${TARGET_COMPILE_FLAGS} ${TARGET_OPTIONS}")
+endif()
+
+include_directories(${PROJECT_SOURCE_DIR})
+
+set(CMAKE_C_FLAGS ${CMAKE_C_FLAGS} ${TARGET_COMPILE_FLAGS})
+set(CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS} ${TARGET_COMPILE_FLAGS})
+set(CMAKE_EXE_LINKER_FLAGS ${CMAKE_EXE_LINKER_FLAGS} ${TARGET_LINKER_FLAGS})
+
+if(ENABLE_PERF_MONITORING)
+    message("Will enable performance monitoring")
+    message(WARNING "Performance monitoring is support only on Linux")
+    message(WARNING "If you're running in a VM, performance monitoring will be broken, "
+                    "and will likely break inference.")
+    add_definitions(-DENABLE_PERF_MONITORING)
+endif()
+
+add_library(xinfer STATIC x_inference_lib.c x_perf_monitor.c)
+
+if(LIB_ONLY)
+    message("Will only build the library")
+else()
+    message("Will build the library and the executable")
+    if (BUILD_FOR_DYNAMIC_LINKAGE)
+        message("Will build the executable for dynamic bundle linkage")
+        set(OBJS )
+        add_definitions(-DX_USE_DYNAMIC)
+    else()
+        if(NOT DEFINED LINKED_BUNDLE)
+            message(FATAL_ERROR "When building with static bundle linkage, bundle must be specified with -DLINKED_BUNDLE=")
+        endif()
+        if (NOT DEFINED LINKED_MODEL_NAME)
+            message(FATAL_ERROR "When building with static bundle linkage, model name must be specified with -DLINKED_MODEL_NAME=")
+        endif()
+
+        add_definitions(-DX_MODEL_NAME=${LINKED_MODEL_NAME})
+        set(OBJS ${LINKED_BUNDLE})
+        set_source_files_properties(${OBJS} PROPERTIES EXTERNAL_OBJECT TRUE GENERATED TRUE)
+    endif()
+    link_directories(${LIBRARY_OUTPUT_PATH})
+    add_executable(x-infer main.c ${OBJS})
+    target_link_libraries(x-infer xinfer dl m)
+endif()