nupic.core Extraction Plan
Clone this wiki locally
NuPIC is currently undergoing a major refactoring, with the goal of extracting a C++ "core" that contains all CLA algorithms. This will provide a core repository so other language bindings can be created aside from Python. The extraction plan for this refactoring is detailed below and kept up-to-date by Matt Taylor.
- One repo:
- primary interface: python
- all examples are python
- all tutorials are python
- underlying C++ interface that no one uses
- C++ is compiled with SWIG with python bindings
nupic.corecontains all C++
- all algorithms are implemented in C++
- complete API documentation
- tutorials for direct usage of C++ API with C++ client
- tutorial for creating language bindings
- python binding and client created from
nupicproject python code
- community is responsible for creating bindings and clients in other environments
- undecided whether language bindings and clients will be combined or in different repos
- undecided naming scheme for bindings and clients, but we should provide guidance with python
Step 1: Initial Split
Summary of initiatives:
nupicinto two repositories
nupicbuild process to accommodate these changes
checkout and build
nupic.coreas a dependency
decide what a passing build means by these new standards
Establish an independent
nupicstart using the
nupic.corebuild instead of building it itself
nupic.core will contain only C++ code with minimal dependencies.
nupic will contain everything else currently within
Clean up existing build script nupic #583 Remove all python traces from nupic in preparation for nupic.core nupic #584 Create nupic.core repository in github nupic #585 Remove C++ code from nupic repository and add nupic.core dependency nupic #587 Ensure nupic python tests pass after nupic.core dependency has been added nupic #588 Create a mechanism to update the nupic.core dependency from nupic with every push to nupic.core master #590
Update nupic build to CMake nupic #647 Travis runs cmake builds in clang & gcc Update README build instructions for cmake Deprecates
build.shprocess in favor of cmake
Remove old build process and artifacts from nupic nupic #673
Establish independent nupic.core build process nupic.core #4
Document the proposed directory structure for nupic.core and nupic [#591](https://github.com/numenta/nupic/issues/591) Run nupic.core build and tests in Travis-CI nupic.core #5 Ensure all C++ tests pass within new core c-only project nupic.core #9
Implement new directory structures
nupicto use the
nupic.corebuild system nupic #625
Step 2: Prepare for
In order to provide a stable
nupic.core for developers to program language bindings and clients against, we need to have a stable
v1.0 release. Requirements for a stable release are defined below.
The following requirements do not affect each other and can be worked upon simultaneously.
1. Define API with Tests
To ensure backwards-compatibility, a new test suite must be created for
nupic.core that exercises all public API endpoints. These tests do not need to be extensive functional tests, but they do need to validate the signatures of the endpoints (both input and output) and assert basic expectations about the structure of the public API. This test suite MUST cover all classes and functions that are expected to be a part of the API.
The API tests exist to establish a contract that can be run in the CI pipeline that will identify breaking API changes in future pull requests. It also makes it obvious for reviewers when a PR modifies an API tests that backwards compatibility is intentionally being broken.
2. Consolidate All CLA Algorithms in
nupic.core MUST contain all CLA algorithms before it can be released, simply because it is incomplete otherwise.
The following necessary components must be translated from Python in
nupic into C++ in
- current "temporal pooler" (sequence memory)
This should not change the
nupic.core API defined by the API tests.
3. Remove Unused Code From
Identify and remove unused code.
Step 3: Release
Creating a stable release fork for
nupic.core is essential for client programmers to create tools for running NuPIC in different environments.
We'll discuss options for the mechanics of this work on the nupic-hackers mailing list at the appropriate time and fill this section in with the results.
Step 4: Create python language binding and client library
Our first official client library will be a conversion of
nupic-py (bindings) and
nupic-py-client (OPF client). This will be a functional example of how to create a client for other programmers of bindings and clients.
Work on this step can be done immediately after Step 1 above is complete. Steps 2/3 above are not prerequisites.
Summary of initiatives:
nupicinto two repositories:
nupic-py: contains only Python language bindings
nupic-py-client: contains OPF, encoders, etc. used by Grok
Update CI process to accommodate three repositories
- Does a
nupic.corechange trigger builds for all repos synchronously?
- How does the bindings repo include the core?
- How does the python client include the bindings?
- Does a