Skip to content
master
Go to file
Code

Latest commit

Summary:
Pull Request resolved: #459

transpose kernel functions for uint8 type:
A 8x32 uint8 transpose kernel
A mxn uint8 transpose kernel for corner cases

Reviewed By: dskhudia

Differential Revision: D24812974

fbshipit-source-id: 75f0c3b4a87a45dcd9492e4465215dc1bec0cbbb
bb111f3

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
src
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

README.md

FBGEMM

Linux Build: CircleCI

FBGEMM (Facebook GEneral Matrix Multiplication) is a low-precision, high-performance matrix-matrix multiplications and convolution library for server-side inference.

The library provides efficient low-precision general matrix multiplication for small batch sizes and support for accuracy-loss minimizing techniques such as row-wise quantization and outlier-aware quantization. FBGEMM also exploits fusion opportunities in order to overcome the unique challenges of matrix multiplication at lower precision with bandwidth-bound operations.

FBGEMM is used as a backend of Caffe2 and PyTorch quantized operators for x86 machines:

What's New?

Examples

The tests (in test folder) and benchmarks (in bench folder) are some great examples of using FBGEMM. For instance, SpMDMTest test in test/PackedRequantizeAcc16Test.cc shows how to combine row offset calculations with packing of A (PackAWithRowOffset), how to pack B matrix (PackBMatrix) and construct output pipeline (sparse_matrix*dense_matrix --> requantization --> nop) fused with inner GEMM macro kernel.

Build Notes

FBGEMM uses the standard CMAKE-based build flow.

Dependencies

FBGEMM requires gcc 5+ and a CPU with support for avx2 instruction set or higher. It's been tested on Mac OS X and Linux.

  • asmjit

With inner kernels, FBGEMM takes a “one size doesn't fit all” approach, so the implementation dynamically generates efficient matrix-shape specific vectorized code using a third-party library called asmjit. asmjit is required to build FBGEMM.

  • cpuinfo

FBGEMM detects CPU instruction set support at runtime using cpuinfo library and dispatches optimized kernels for the detected instruction set. Therefore, cpuinfo is required to detect CPU type.

  • googletest

googletest is required to build and run FBGEMM's tests. googletest is not required if you don't want to run FBGEMM tests. By default, building of tests is on. Turn it off by setting FBGEMM_BUILD_TESTS to off.

You can download asmjit, cpuinfo, googletest and set ASMJIT_SRC_DIR, CPUINFO_SRC_DIR, GOOGLETEST_SOURCE_DIR respectively for cmake to find these libraries. If any of these variables is not set, cmake will build the git submodules found in the third_party directory.

FBGEMM, in general, does not have any dependency on Intel MKL. However, for performance comparison, some benchmarks use MKL functions. If MKL is found or MKL path is provided with INTEL_MKL_DIR benchmarks are built with MKL and performance numbers are reported for MKL functions as well. However, if MKL is not found, the benchmarks are not built.

General build instructions are as follows:

git clone --recursive https://github.com/pytorch/FBGEMM.git
cd FBGEMM
# if you are updating an existing checkout
git submodule sync
git submodule update --init --recursive
mkdir build && cd build
cmake ..
make

To run the tests after building FBGEMM (if tests are built), use the following command:

make test

Installing FBGEMM

make install

How FBGEMM works

For a high-level overview, design philosophy and brief descriptions of various parts of FBGEMM please see our blog.

Full documentation

We have extensively used comments in our source files. The best and up-do-date documentation is available in the source files.

Join the FBGEMM community

See the CONTRIBUTING file for how to help out.

License

FBGEMM is BSD licensed, as found in the LICENSE file.

About

FB (Facebook) + GEMM (General Matrix-Matrix Multiplication) - https://code.fb.com/ml-applications/fbgemm/

Resources

License

Releases

No releases published

Packages

No packages published

Languages

You can’t perform that action at this time.