C++ dynamic ndarray library, with Python exposure
C++ CMake Jupyter Notebook Other
Latest commit 7bec433 Aug 29, 2016 @mwiebe mwiebe committed on GitHub Merge pull request #1313 from mwiebe/callable_parse
Move callable_type fully into libdynd from libdyndt
Permalink
Failed to load latest commit information.
benchmarks A little clean-up, benchmarks fix May 13, 2016
buildscripts Update copyright notices to 2015 Jan 16, 2015
cmake Untabify Mar 16, 2016
conda.recipe Re-add code using script_env in meta.yaml to forward CC and CXX to Jun 24, 2016
devdocs Save changes Jul 17, 2016
docs Move docs about developing DyND away from docs about using DyND May 2, 2016
examples Update simple_example.cpp Jul 13, 2016
include/dynd Reenable callable_type typevar substitution via generic type reconstr… Aug 29, 2016
plugin Update license notification for 2016. Apr 1, 2016
plugins/mkl Removed unnused variable Jul 16, 2016
src/dynd Fix for warning about unused global Aug 29, 2016
tests Move more type parsing to use the type_registry mechanism Aug 11, 2016
thirdparty Removed cephes Feb 28, 2016
.binstar.yml Update .binstar.yml Feb 19, 2016
.clang-format Move init.hpp from dynd to dyndt, rename to buffer_init.hpp Jul 31, 2016
.gitignore Update notebook with workaround to centered markdown svg link display Dec 6, 2015
.gitmodules Added Google Benchmark as a submodule Oct 6, 2015
.mailmap Add .mailmap file to consolidate author emails Apr 15, 2014
.travis.yml Removed const from single in init_kernel Jul 20, 2016
BUILD.md Add the 'git clone' and Windows instructions to BUILD.md Apr 21, 2016
CMakeLists.txt Change all the parsed callable types to explicit ndt::make_type Aug 28, 2016
LICENSE.md Update copyright notices to 2015 Jan 16, 2015
README.md Update README.md Apr 19, 2016
TESTING.md Fix a typo in TESTING.md. Apr 1, 2016
acling Tidy the docs and structure up a little bit Aug 13, 2012
appveyor.yml Update appveyor.yml May 13, 2016
libdynd-config.bat.in Change nd/ndt to the official names. May 12, 2016
libdynd-config.in Change nd/ndt to the official names. May 12, 2016

README.md

DyND Logo

The DyND Library

Travis CI: Build Status AppVeyor: Build status Coveralls: Coverage Status Gitter: Join the chat at https://gitter.im/libdynd/libdynd

DyND is a dynamic array library for structured and semi-structured data, written with C++ as a first-class target and extended to Python with a lightweight binding. It aims to be a cross-language platform for data analysis, by bringing the popularity and flexibility of the Python data science stack to other languages. It is inspired by NumPy, the Python array programming library at the core of the scientific Python stack, but tries to address a number of obstacles encountered by some of NumPy’s users. Examples of these are support for variable-sized strings, missing values, variable-sized (ragged) array dimensions, and versatile tools for creating functions that apply generic patterns across arrays.

At a high level, the cornerstones of DyND are its type system, array container, and callable (function) objects. These represent the description, storage, and manipulation of dynamic, reinterpretable bytes across languages. At a low level, DyND defines an primitive execution kernel that brings together computation and data in a compact form that can be executed rapidly across array elements. Where DyND begins to shine is in its support for functional composition. For example, NumPy-like broadcasting is not built into every callable, rather it is made available as a functional transformation applied to a scalar callable.

DyND was created by Irwin Zaid and Mark Wiebe. The core team consists of Irwin Zaid, Mark Wiebe, and Ian Henriksen. Others who made important contributions include Phillip Cloud, Michael Droettboom, Stefan Krah, Travis Oliphant, and Andy Terrel. Much of the funding that made this project possible came through Continuum Analytics and DARPA-BAA-12-38, part of XDATA.

We pronounce DyND as "dined", though others refer to it as "dy-n-d". It's not something we're picky about it.

Getting Started

This library is actively developed together with its Python bindings. The Python bindings provide a good way to become familiar with the library from a high level perspective. See the github site for the Python bindings.

C++ is a first-class target of the library, the intent is that all features should be easily usable in that language. This approach makes it so that DyND can expose a more uniform interface to C++, Python, and hopefully other languages that eventually get bindings to the core DyND library.

DyND is still experimental, so many of the interfaces provided here will continue to change. That said, feedback and bug reports are greatly appreciated.

To get started as a developer of libdynd, begin by working through the LibDyND Developer Guide. To discuss the development of this library, subscribe to the LibDyND Development List.

History Of DyND

DyND was started as a personal project of Mark Wiebe in September 2011 to begin dabbling in ideas for how a dynamic, multidimensional array library could be structured in C++. See here for the very first commit. Mark was at the University of British Columbia, then joined Continuum Analytics part-time when it was founded in January 2012, and later became full-time in the spring of 2012. He parted ways with Continuum at the end of 2014, joining Thinkbox Software.

During the formation of Continuum, DyND was open-sourced and brought into the company as a part of the Blaze project. Continuum secured funding for the Blaze project through DARPA's XDATA program, giving the project space and time to develop as it needed and providing real data sets and challenge problems to tackle and measure against. Travis Oliphant has told the story of the early days at Continuum, and why it supports DyND, on the NumPy mailing list - see here.

Irwin Zaid joined the project at the beginning of 2014, as its first major outside contributor, while he was a research fellow at Christ Church, University of Oxford. He initially added CUDA support to the library, then played a gradually increasing role in the design and goals of the project. He took over development of DyND in the spring of 2015, and was funded by Continuum from June 2015 until April 2016.

Ian Henriksen began working on DyND through Google Summer of Code - under the umbrella of NumFocus - in the summer of 2015. He remained active in the project from then onwards.

DyND is still experimental. Some of its components are mature while others are severely lacking. New contributors are welcome, so if you have the patience to collaborate on a maturing code base, and enjoy C++, array-oriented, and numeric programming, DyND might be the the open source project you're looking for.

Building

The build system of this library is based on CMake. See the build instructions for details about how to build the library.

DyND requires a C++14 compiler, the minimum versions supported are gcc 4.9, MSVC 2015, and Clang 3.4. The last release to support C++98 was DyND 0.6.6. C++14 brings several things to the library, including a roughly factor of two compile time improvement and generality with variadic templates.

Documentation

Documentation Index

Running The Tests

The tests are built using Google Test. To execute the test suite, run the test_libdynd program.