Skip to content
Permalink
Browse files

Generic model builder and inference engine (#3415)

Summary:
Quite a bit to chew on in this PR. The main features are:

- Generic model builder (x-model-builder)
- Generic inference engine (x-infer)
- Documentation for the generic model builder and the generic inference engine
Pull Request resolved: #3415

Reviewed By: opti-mix

Differential Revision: D17910520

Pulled By: jackm321

fbshipit-source-id: 44501ee29a32cde260069050776eaccd9eb039b7
  • Loading branch information...
wyessen authored and facebook-github-bot committed Oct 15, 2019
1 parent ef39067 commit d9240b1c842f342fbf3fb2e508b5a0f65aca8911

Large diffs are not rendered by default.

@@ -0,0 +1,44 @@
## XModelRunner

Glow supplies a number of out-of-the-box model builders/runners, which include
the `ImageClassifier` and the `ModelLoader`. The former is tailored towards specific
networks built for image classification, while the latter is a generic model loader (see
the corresponding documentation).

`XModelRunner` is a generic model builder/runner that is able to consume any model --
either in the ONNX format, or as a Caffe2 protobuf -- and
either compile it (producing a bundle if requested, along with profiling when requested),
or run inference on it. `XModelRunner`
is built along with the other models using the standard build instructions (see the
building documentation). Below is a description of the command line options for `XModelRunner`.

### Command Line Options

In addition to all of the "standard" command line options (i.e. those that are common to
all model builders/runners, and can be obtained with the `-help` option), a few
options are supported by `XModelRunner`, some of which are mandatory and some are optional.

#### Required Options

| Option | Expected Values | Description | Required |
| :------ | :------ | :------ | :------ |
|`input-tensor-dims` | Comma-separated list of ints | Input tensor dimensions | Yes |
|`output-tensor-names` | Comma-separated list of strings (no spaces) | Output tensor names (can be more than one) | Yes |
|`write-output` | Boolean flag | Whether to write output to output files (only applicable when not building a bundle) | No (default is `False`) |

#### Expected Input

`XModelRunner` expects either a list of file names containing input (one input tensor per file), which are positional arguments, or a file name containing the list of file names that contain input (one file name per line), specified with the `input-file-list` option. Input specification is required when either profiling the network, or running inference (i.e. not building a bundle). Otherwise, input can be omitted.

#### Produced Output

When input files are specified, output is produced (saved into binary files) only when the `write-output` option is specified. In this case, the input file name acts as a base name for the output file name. The output file name is composed as `[input file name].out.dat`. When input is not specified and bundles are requested, the runner produces bundles in the specified output directory. Also, `write-output` may be omitted when profiling the network.

### Technical Notes

1. The runner currently supports only one (named) input tensor, but can work with multiple (named) output tensors.
2. The runner does not currently support streaming input, but this feature is planned.

### A Note on the Initial Contribution

The initial contribution of this runner (as well as the corresponding documentation) was made as part of the open source contribution initiative by the XPERI Corporation (xperi.com).
@@ -0,0 +1,70 @@
cmake_minimum_required(VERSION 3.0)
project(x-infer)

option(LIB_ONLY "Only build the library" OFF)
option(BUILD_FOR_DYNAMIC_LINKAGE "Whether to build for dynamic or static bundle linkage" ON)
option(LINK_LIBS_STATICALLY "Whether to link in all the required libraries statically" ON)
option(ENABLE_PERF_MONITORING "Whether to enable performance monitoring" OFF)

set(EXECUTABLE_OUTPUT_PATH ${CMAKE_BINARY_DIR}/bin)
set(LIBRARY_OUTPUT_PATH ${CMAKE_BINARY_DIR}/bin)

if(LINK_LIBS_STATICALLY)
message(WARNING "Static linking may cause issues with dlopen on target")
message("Will statically link executable")
set(TARGET_COMPILE_FLAGS "-static"
)
set(TARGET_LINKER_FLAGS "-static"
)
else()
set(TARGET_COMPILE_FLAGS )
endif()

if (DEFINED TARGET_OPTIONS)
set(TARGET_COMPILE_FLAGS "${TARGET_COMPILE_FLAGS} ${TARGET_OPTIONS}")
endif()

include_directories(${PROJECT_SOURCE_DIR})

set(CMAKE_C_FLAGS ${CMAKE_C_FLAGS} ${TARGET_COMPILE_FLAGS})
set(CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS} ${TARGET_COMPILE_FLAGS})
set(CMAKE_EXE_LINKER_FLAGS ${CMAKE_EXE_LINKER_FLAGS} ${TARGET_LINKER_FLAGS})

if(ENABLE_PERF_MONITORING)
message("Will enable performance monitoring")
message(WARNING "Performance monitoring is support only on Linux")
message(WARNING "If you're running in a VM, performance monitoring will be "
"broken, and will likely break inference.")
add_definitions(-DENABLE_PERF_MONITORING)
endif()

add_library(xinfer STATIC x_inference_lib.c x_perf_monitor.c)

if(LIB_ONLY)
message("Will only build the library")
else()
message("Will build the library and the executable")
if (BUILD_FOR_DYNAMIC_LINKAGE)
message("Will build the executable for dynamic bundle linkage")
set(OBJS )
add_definitions(-DX_USE_DYNAMIC)
else()
if(NOT DEFINED LINKED_BUNDLE)
message(FATAL_ERROR "When building with static bundle linkage, "
"bundle must be specified with "
"-DLINKED_BUNDLE=")
endif()
if (NOT DEFINED LINKED_MODEL_NAME)
message(FATAL_ERROR "When building with static bundle linkage, "
"model name must be specified with "
"-DLINKED_MODEL_NAME=")
endif()

add_definitions(-DX_MODEL_NAME=${LINKED_MODEL_NAME})
set(OBJS ${LINKED_BUNDLE})
set_source_files_properties(${OBJS} PROPERTIES EXTERNAL_OBJECT TRUE GENERATED TRUE)
endif()
link_directories(${LIBRARY_OUTPUT_PATH})
add_executable(x-infer main.c ${OBJS})
target_link_libraries(x-infer xinfer dl m)
endif()

0 comments on commit d9240b1

Please sign in to comment.
You can’t perform that action at this time.