/arrow

Apache Arrow is a cross-language development platform for in-memory data. It specifies a standardized language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware. It also provides computational libraries and zero-copy streaming messaging and interprocess communication…
  1. C++ 43.6%
  2. Java 14.6%
  3. Python 9.5%
  4. Rust 7.8%
  5. TypeScript 5.5%
  6. Ruby 3.8%
  7. Other 15.2%
Find file
Clone or download

Clone with HTTPS

Use Git or checkout with SVN using the web URL.

Open in Desktop Download ZIP

Launching GitHub Desktop...

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop...

If nothing happens, download GitHub Desktop and try again.

Launching Xcode...

If nothing happens, download Xcode and try again.

Launching Visual Studio...

If nothing happens, download the GitHub extension for Visual Studio and try again.

@kou @pcmoritz
kou and pcmoritz ARROW-4455: [Plasma] Suppress class-memaccess warnings 
    cpp/src/plasma/store.cc: In member function 'arrow::Status plasma::PlasmaStore::ProcessMessage(plasma::Client*)':
    cpp/src/plasma/store.cc:767:36: error: 'void* memset(void*, int, size_t)' clearing an object of type 'struct plasma::PlasmaObject' with no trivial copy-assignment; use assignment or value-initialization instead [-Werror=class-memaccess]
       memset(&object, 0, sizeof(object));
                                        ^
    In file included from cpp/src/plasma/eviction_policy.h:27,
                     from cpp/src/plasma/store.h:30,
                     from cpp/src/plasma/store.cc:29:
    cpp/src/plasma/plasma.h:75:8: note: 'struct plasma::PlasmaObject' declared here
     struct PlasmaObject {
            ^~~~~~~~~~~~
    cpp/src/plasma/test/serialization_tests.cc: In function 'plasma::PlasmaObject plasma::random_plasma_object()':
    cpp/src/plasma/test/serialization_tests.cc:68:36: error: 'void* memset(void*, int, size_t)' clearing an object of type 'struct plasma::PlasmaObject' with no trivial copy-assignment; use assignment or value-initialization instead [-Werror=class-memaccess]
       memset(&object, 0, sizeof(object));
                                        ^
    In file included from cpp/src/plasma/test/serialization_tests.cc:25:
    cpp/src/plasma/plasma.h:75:8: note: 'struct plasma::PlasmaObject' declared here
     struct PlasmaObject {
            ^~~~~~~~~~~~
    cpp/src/plasma/test/serialization_tests.cc: In member function 'virtual void plasma::PlasmaSerialization_CreateReply_Test::TestBody()':
    cpp/src/plasma/test/serialization_tests.cc:110:38: error: 'void* memset(void*, int, size_t)' clearing an object of type 'struct plasma::PlasmaObject' with no trivial copy-assignment; use assignment or value-initialization instead [-Werror=class-memaccess]
       memset(&object2, 0, sizeof(object2));
                                          ^
    In file included from cpp/src/plasma/test/serialization_tests.cc:25:
    cpp/src/plasma/plasma.h:75:8: note: 'struct plasma::PlasmaObject' declared here
     struct PlasmaObject {
            ^~~~~~~~~~~~

Author: Kouhei Sutou <kou@clear-code.com>

Closes #3543 from kou/plasma-suppress-class-memaccess-warning and squashes the following commits:

34ce66c <Kouhei Sutou>  Suppress class-memaccess warnings
Latest commit f08e109 Feb 1, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github ARROW-2907: [GitHub] Improve the first paragraph of "How to contribut… Jul 25, 2018
c_glib ARROW-4235: [GLib] Use "column_builder" in GArrowRecordBatchBuilder Feb 1, 2019
ci ARROW-4277: [C++] Add gmock to the toolchain Feb 1, 2019
cpp ARROW-4455: [Plasma] Suppress class-memaccess warnings Feb 2, 2019
csharp ARROW-4284: [C#] File / Stream serialization fails due to type mismat… Jan 23, 2019
dev ARROW-4358: [CI] Restore support for trusty in CI Jan 28, 2019
docs ARROW-4274: [C++][Gandiva] split decimal into two parts Jan 24, 2019
format ARROW-4366: [Docs] Change extension from format/README.md to format/R… Jan 25, 2019
go ARROW-4126: [Go] offset not used when accessing boolean array Jan 7, 2019
integration ARROW-4213: [Flight] Fix incompatibilities between C++ and Java Jan 29, 2019
java ARROW-3965 [Java] JDBC-To-Arrow Configuration Jan 30, 2019
js include new jest configs to ensure verify release script works (#3532) Jan 31, 2019
matlab [Release] Update versions for 0.13.0-SNAPSHOT Jan 19, 2019
python ARROW-4277: [C++] Add gmock to the toolchain Feb 1, 2019
r ARROW-4328: Add a ARROW_USE_OLD_CXXABI configure var to R Jan 30, 2019
ruby ARROW-4235: [GLib] Use "column_builder" in GArrowRecordBatchBuilder Feb 1, 2019
rust ARROW-4403 : [Rust] Fix format errors Jan 28, 2019
site ARROW-4368: [Docs] Fix install document for Ubuntu 16.04 or earlier Jan 28, 2019
.clang-format ARROW-3313: [R] Move .clang-format to top level. Add r/lint.sh script… Sep 26, 2018
.clang-tidy ARROW-3313: [R] Move .clang-format to top level. Add r/lint.sh script… Sep 26, 2018
.clang-tidy-ignore ARROW-3313: [R] Move .clang-format to top level. Add r/lint.sh script… Sep 26, 2018
.dockerignore ARROW-3834: [Doc] Merge C++ and Python documentation Dec 6, 2018
.gitattributes ARROW-3282: [R] initial R functionality Sep 25, 2018
.gitignore ARROW-4281: [CI] Use Ubuntu Xenial VMs on Travis-CI Jan 23, 2019
.gitmodules ARROW-3075: [C++] Incorporate parquet-cpp codebase into Arrow C++ build Sep 7, 2018
.pre-commit-config.yaml ARROW-3834: [Doc] Merge C++ and Python documentation Dec 6, 2018
.readthedocs.yml ARROW-1142: [C++] Port over compression toolchain and interfaces from… Jun 23, 2017
.travis.yml [Python] Use 'latest' manylinux1 build image Feb 1, 2019
CHANGELOG.md [Release] Update CHANGELOG.md for 0.12.0 Jan 16, 2019
CODE_OF_CONDUCT.md ARROW-4006: Add CODE_OF_CONDUCT.md Dec 15, 2018
LICENSE.txt ARROW-4198: [Gandiva] Added support to cast timestamp Jan 31, 2019
NOTICE.txt ARROW-249: [JAVA] Flight GRPC Implementation Sep 25, 2018
README.md [Gandiva] Add link to Gandiva codebase in top level README Nov 15, 2018
appveyor.yml ARROW-3846: [Gandiva][C++] Build Gandiva C++ libraries and get unit t… Feb 1, 2019
docker-compose.yml ARROW-4401: [Python] Alpine dockerfile fails to build because pandas … Jan 29, 2019
header ARROW-259: Use Flatbuffer Field type instead of MaterializedField Aug 18, 2016

README.md

Apache Arrow

Build Status travis build status Code Coverage codecov.io code coverage

Powering In-Memory Analytics

Apache Arrow is a development platform for in-memory analytics. It contains a set of technologies that enable big data systems to process and move data fast.

Major components of the project include:

Arrow is an Apache Software Foundation project. Learn more at arrow.apache.org.

What's in the Arrow libraries?

The reference Arrow libraries contain a number of distinct software components:

  • Columnar vector and table-like containers (similar to data frames) supporting flat or nested types
  • Fast, language agnostic metadata messaging layer (using Google's Flatbuffers library)
  • Reference-counted off-heap buffer memory management, for zero-copy memory sharing and handling memory-mapped files
  • Low-overhead IO interfaces to files on disk, HDFS (C++ only)
  • Self-describing binary wire formats (streaming and batch/file-like) for remote procedure calls (RPC) and interprocess communication (IPC)
  • Integration tests for verifying binary compatibility between the implementations (e.g. sending data from Java to C++)
  • Conversions to and from other in-memory data structures

Getting involved

Even if you do not plan to contribute to Apache Arrow itself or Arrow integrations in other projects, we'd be happy to have you involved:

How to Contribute

We prefer to receive contributions in the form of GitHub pull requests. Please send pull requests against the github.com/apache/arrow repository.

If you are looking for some ideas on what to contribute, check out the JIRA issues for the Apache Arrow project. Comment on the issue and/or contact dev@arrow.apache.org with your questions and ideas.

If you’d like to report a bug but don’t have time to fix it, you can still post it on JIRA, or email the mailing list dev@arrow.apache.org

To contribute a patch:

  1. Break your work into small, single-purpose patches if possible. It’s much harder to merge in a large change with a lot of disjoint features.
  2. Create a JIRA for your patch on the Arrow Project JIRA.
  3. Submit the patch as a GitHub pull request against the master branch. For a tutorial, see the GitHub guides on forking a repo and sending a pull request. Prefix your pull request name with the JIRA name (ex: https://github.com/apache/arrow/pull/240).
  4. Make sure that your code passes the unit tests. You can find instructions how to run the unit tests for each Arrow component in its respective README file.
  5. Add new unit tests for your code.

Thank you in advance for your contributions!