Skip to content
A framework to integrate FPGA accelerators with Apache Arrow
Branch: develop
Clone or download
johanpel Update examples (#132)
* [Fletchgen] Exit on invalid command line options

* Rename sim_top to sim_top_tc for vhdeps

* Fix a bug in EPC propagation

* Update READMEs to reflect Fletchgen changes
Latest commit 67ce518 Jun 25, 2019

README.md

Fletcher : A framework to integrate FPGA accelerators with Apache Arrow

Build Status Pipeline Status License Last Commit

Fletcher is a framework that helps to integrate FPGA accelerators with tools and frameworks that use Apache Arrow in their back-ends.

Apache Arrow specifies an in-memory format for data and provides libraries to various languages to interface with the data in that format. The format prevents the need for serialization and through the libraries Arrow, zero-copy interprocess communication is possible. Languages that have Arrow libraries (under development) include C, C++, Go, Java, JavaScript, Python, Ruby and Rust.

While many software projects can benefit from these advantages, also hardware accelerated applications have seen serious serialization bottlenecks. Fletcher focuses on FPGA accelerators. Through Fletcher and Arrow, interfacing efficiently between FPGA accelerator and language runtime is made available to all the supported languages.

Given an Arrow Schema (description of a tabular datastructure), Fletcher generates the following:

  • An easy-to-use hardware interface for the functional part of the accelerator:
    • You provide a range indices of your Arrow RecordBatch (rather than byte address)
    • You receive streams of the data-type specified by the schema (rather than bus words)
    • No pointer arithmetic, reordering, buffering, etc.. required - Fletcher does this for you, with high throughput.
  • A template for the functional part of accelerator (to be implemented manually or using high-level synthesis)
    • You connect directly to streams of data from/to your RecordBatch rather than some memory bus interface.

Current state

Our framework is functional, but in early stages of development.

Especially the development branch (which is currently our main branch) is very active and may break without notice. The (platform-specific) examples are quite hard to integrate in a CI pipeline (they would take multiple days to complete and would incur significant costs for platforms such as Amazon's EC F1), so it's best to test them out using the master branch. The master branch can be rather old, but will be in a state where all examples should work on all supported platforms.

Apache Arrow support

  • Fletcher currently supports:

    • Reading/writing an Arrow Schema created from any combination of:
      • Fixed-width primitives
      • Lists
      • Structs
  • Validity bitmaps.

  • We are working on support for:

    • Sparse and dense unions
    • Dictionaries
    • Chunked tabular structures (Arrow::Table)

Platform support

  • Our core hardware descriptions are vendor independent; we don't use any vendor IP.

    • You can simulate a Fletcher based design without a specific target platform.
    • Tested simulators include Questa/Modelsim, GHDL and XSIM.
  • We provide top-level wrappers for the following platforms:

    • Amazon EC2 F1
    • OpenPOWER CAPI SNAP
    • Our top-level can be generated to speak AXI, so it should be easy to integrate with many existing systems that provide:
      • AXI4 (full) interface to memory, and
      • AXI4-lite interface for MMIO.

Further reading

Example projects

External projects using Fletcher:

You can’t perform that action at this time.