The core Holochain framework written in Rust, a container API for that, and hdk-rust library for Zomes
Switch branches/tags
79-persistence 103-web-container-interface 188-entry-meta 254-dpki-adr 350-refactor-pure-reducers 586-state-refactor 2018-11-10-dna-entry-move 2018-11-10-entry-type-move 2018-11-10-nix-mac-os-x 2018-11-15-make-refactor 2018-11-17-fix-warnings 2018-11-17-spike-travis-nix 2018-11-18-merge-container-api-wasm 2018-11-19-blacklist-incremental-travis-cache 2018-11-19-merge-debug-wasm 2018-11-19-reinstate-get-link-result 2018-11-19-validation-package-wasm-merge 2018-11-21-reinstate-get-entry-result 2018-11-21-wasm-workspace 2018-11-26-nix-build-commands 2018-11-26-unwrap-cleanup 2018-11-27-unwrap-cleanup 2018-11-29-travis-windows-cache 2018-12-02-travis-nix 2018-12-11-migrate-entry-basic 2018-12-14-nix-node-test 2018-12-15-circle-wasm-cache ADR-14 add-zmq-dep-to-readme additional-container-api-getters bridges-dna-definition capabilities cargo-make cmd-deploy-test context-refactoring core-api-merge damien-error develop docker-ubuntu-fix eav-attribute-name-validate emalinus-patch-1 entry-trait entryenum feature/call get-links-and-load gh-pages hc-run-docs hc-watch implement-hdk-call initialize-network-on-load js-test-scaffold-version-mismatch link-roundtrip-test link-validation master mdbook-push mdbook-updates net-ipc-zmq-test paritytech-websocket pr-620/holochain/588-define-zome-jsonstring pr-622/holochain/586-state-refactor pr-631/holochain/networking-via-context refactor-sodium-pkg run-dna-with-optional-context rust-edition-2018 rust-scaffold-fixes scenario-waitfor signals-bugfix test-coverage test-macro-links-fix update-send-docs version-dep-fixes zmq_win_fix
Nothing to show
Clone or download
thedavidmeister Merge pull request #757 from holochain/2018-12-14-nix-node-test
move node container test to circle and remove clean step
Latest commit 4c7492c Dec 14, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.circleci lint Dec 14, 2018
app_spec merge develop Dec 14, 2018
c_binding_tests/hc_dna Allow building c_binding_tests/hc_dna using homebrew Qt on Mojave. Oct 17, 2018
cas_implementations Merge branch 'develop' into 2018-11-20-entry-enum Dec 3, 2018
cmd Merge branch 'develop' into signals-internal Dec 14, 2018
container Remove signal handler reference in `container` Dec 14, 2018
container_api Uniquify test DNA Dec 14, 2018
core Merge remote-tracking branch 'origin/develop' into signals-internal Dec 14, 2018
core_api_c_binding merge from develop Nov 29, 2018
core_types Merge remote-tracking branch 'origin/develop' into signals-internal Dec 14, 2018
core_types_derive version bump for 0.0.2 Nov 27, 2018
dna_c_binding Revert "remove all tarpaulin skips" Nov 30, 2018
doc Merge pull request #385 from holochain/ADR-14 Dec 12, 2018
docker faster docker build Dec 13, 2018
hdk-rust Merge remote-tracking branch 'origin/develop' into signals-internal Dec 14, 2018
net small lint Dec 14, 2018
net_connection rustfmt Dec 13, 2018
net_ipc minor rename Dec 13, 2018
nodejs_container lint Dec 14, 2018
rust_sodium-sys test coverage Dec 4, 2018
scripts debug travis Dec 12, 2018
sodium fmt Dec 5, 2018
test_bin rustfmt hell Dec 14, 2018
test_utils Merge remote-tracking branch 'origin/develop' into signals-internal Dec 14, 2018
vendor/zmq some cleanup Nov 7, 2018
wasm_utils merge develop Dec 13, 2018
.adr-dir Imported ADRs that Damian shared via gdrive May 16, 2018
.dockerignore tweak circle cache paths Dec 13, 2018
.gitignore Added more in gitignore Dec 10, 2018
.gitmodules step2 - delete old code prep for net_connection Nov 9, 2018
.travis.yml move node container test to circle and remove clean step Dec 14, 2018
Cargo.toml removed addition of app_spec/zomes/blog/code from Cargo.toml Dec 6, 2018
Dockerfile.circle lint Dec 13, 2018
LICENSE added license Jun 15, 2018
Makefile sync tools version Dec 13, 2018
README.md [skip travis] fix link Nov 28, 2018
_config.yml add deploy-docs.sh Aug 7, 2018
build-circle WIP on passing tests in circle Dec 13, 2018
build.bat fixed get entry tests + build.bat Nov 22, 2018
codecov.yml drop coverage to 70 Dec 12, 2018
rustfmt.toml rustfmt -> edition 2018 Nov 27, 2018
shell.nix do less in circle Dec 14, 2018

README.md

Holochain-rust

holochain logo

Project PM Chat

Twitter Follow

Travis Codecov In Progress License: GPL v3

This is the home of the Holochain Rust libraries, being rewritten from Go into Rust, and extended.

Code Status: Rust version is currently Pre-Alpha. Not for production use. The code has not yet undergone a security audit. We expect to destructively restructure code APIs and data chains until Beta. Prototype go version was unveiled at our first hackathon (March 2017), with go version Alpha 0 released October 2017. Alpha 1 was released May 2018. We expect a developer pre-release of this Rust re-write in mid October 2018.

Holochain Links: FAQ Developer Docs White Paper

Overview

This holochain-rust repository implements a number of distinct yet overlapping aspects of the Holochain framework.

  1. A library for the core Holochain functionality for defining and running DNA instances: core
  2. A library and syntax for use in Rust based development of Zomes within DNAs, called Holochain Development Kit: hdk-rust
  3. A library for managing instances and connecting them to interfaces: container_api
  4. A Rust based container that uses the container_api: container
  5. A nodejs based container for running tests: container
  6. A command line developer tool: hc
  7. A sample application that we use to demonstrate the current functionality and drive development: app-spec

Let's elaborate on these three aspects.

Core

The core folder contains the code that implements the core functionality of Holochain. It is the aspect that takes in an application DNA, and an agent, and securely runs peer-to-peer applications by exposing the API functions to Zomes. It draws on other top level definitions and functions such as dna, cas_implementations, agent, and core_types.

HDK Rust

Holochain applications have been designed to consist at the low-level of WebAssembly (WASM) running in a virtual machine environment. However, most languages will be easiest to use with some stub code to connect into the WASM runtime environment, because of some constraints with WASM. That is the main reason why a "Developer Kit" for a language exists. It is a library, and a syntax, for writing Zome code in that language.

hdk-rust is a solid reference implementation of this, that enables Zomes to be written in the Rust language (the same, somewhat confusingly, as Holochain Core).

Within this repository, some aspects cross over between core and hdk-rust, such as core_types, since they get stored into WASM memory in core, and then loaded from WASM memory, within hdk-rust. Related, wasm_utils is used on both sides to actually perform the storing, and loading, of values into and from WASM memory.

Other HDKs and language options

Any language that compiles to WASM and can serialize/deserialize JSON data can be available as an option for programmers to write Holochain applications.

An HDK for Assemblyscript is under experimental development at hdk-assemblyscript.

We expect many more languages to be added by the community, and there is even an article on how to write a kit for a new language.

Container API

Core only implements the logic for the execution of a single application. Because the Holochain app ecosystem relies on DNA composibility, we need to be able to load and instantiate multiple DNAs. We call an executable that can do this an container. The first such containers we implemented were the GUI driven holosqape and the CLI driven hcshell container which we used for running javascript based tests.

These gave us the experience from which we abstracted the container_api crate which specifies and implements a standard way for building containers, including specifying the various interfaces that might be available for executing calls on a particular DNA, i.e. websockets, HTTP, Unix domain sockets, carrier pigeon network, etc...

If you need to implement your own container, container_api should provide you with the needed types and functions to do so easily.

To implement a container in a C based language, the core_api_c_binding [NEEDS UPDATING] code could be used, such as HoloSqape does.

Rust Container

The container crate uses the container_api to implement an executable which is intended to become the main, highly configurable and GUI less container implementation that can be run as a background system service. Currently the Rust Container

Nodejs Container

The nodejs_container directory implements a node package that creates a container that wraps the Holochain core Rust implementation so we can access it from node. This is crucial especially for creating a test-driven development environment for developing Holochain DNA. The hc command-line tool relies on it to run tests.

HC Command-line developer tool.

The cmd crate implements our command line developer tool which allows you to create DNA scaffold, run tests, and finally package your DNA for running in a containter. For more details see the crate README.

Documentation: The Book on Holochain

There is a work-in-progress book of documentation being written about holochain-rust. See the published version at the associated GitHub Pages for this repo, https://developer.holochain.org/guide/latest. See instructions for how to contribute to the book at doc/holochain_101/src/how_to_contribute.md.

Installation & Usage

Important: for installation of the tools with which you can build Holochain applications, you will want to instead proceed to the instructions on the quick start installation guide.

https://developer.holochain.org/start.html

The following instructions are for developing Holochain Core or the HDK itself

There are two approaches to building and testing Holochain, using make or using docker:

Make (ubuntu and macOS only)

If you are running on ubuntu or macOS, and you have make installed, you can do local development by simply typing:

make which will:

  1. install (or update to) the correct version of rust
  2. build all the rust libraries from the source code in this repository.
  3. build and install the command-line tools.

Docker

We also use docker. The docker folder contains scripts to build and run docker images.

NixOS

If you have nix-shell then feel free to use our .nix files.

shell.core.nix and shell.tools.nix are split to mirror the versioning behaviour in the makefile.

Not everything in the Makefile is implemented in nix, and a lot of things don't need to be. Notably the cross-platform and defensive installation of dependencies is not included.

If you have a nix friendly system, this is probably the fastest way to develop and test.

e.g. nix-shell shell.core.nix --run "hc-wasm-build && hc-test"

Running tests

Run:

. docker/run-test

or

make test

Note that there are also make commands for running the tests of just core, or the command-line line tools or app_spec separately:

make test_cmd
make test_holochain
make test_app_spec
make build_nodejs_container

Code style

There is a linter/formatter enforcing code style.

Run:

. docker/run-fmt

or

make fmt

Compiler warnings

Compilation warnings are NOT OK in shared/production level code.

Warnings have a nasty habit of piling up over time. This makes your code increasingly unpleasant for other people to work with.

CI MUST fail or pass, there is no use in the ever noisier "maybe" status.

If you are facing a warning locally you can try:

  1. Fixing it
  2. Using #[allow(***)] inline to surgically override a once-off issue
  3. Proposing a global allow for a specific rule
  • this is an extreme action to take
  • this should only be considered if it can be shown that:
    • the issue is common (e.g. dozens of #allow[***])
    • disabling it won't cause issues/mess to pile up elsewhere
    • the wider Rust community won't find our codebase harder to work with

If you don't know the best approach, please ask for help!

It is NOT OK to disable deny for warnings globally at the CI or makefile/nix level.

You can allow warnings locally during development by setting the RUSTFLAGS environment variable.

CI configuration changes

Please also be aware that extending/changing the CI configuration can be very time consuming. Seemingly minor changes can have large downstream impact.

Some notable things to watch out for:

  • Adding changes that cause the Travis cache to be dropped on every run
  • Changing the compiler/lint rules that are shared by many people
  • Changing versions of crates/libs that also impact downstream crates/repos
  • Changing the nightly version of Rust used
  • Adding/removing tools or external libs

The change may not be immediately apparent to you. The change may break the development environment on a different operating system, e.g. Windows.

At the same time, we do not want to catastrophise and stifle innovation or legitimate upgrades.

If you have a proposal to improve our CI config, that is great!

Please open a dedicated branch for the change in isolation so we can discuss the proposal together.

Please broadcast the proposal in chat to maximise visibility and the opportunity for everyone to respond.

It is NOT OK to change the behaviour of tests/CI in otherwise unrelated PRs. SOMETIMES it MAY be OK to change CI in a related PR, e.g. adding a new lib that your code requires. DO expect that a change like this will probably attract additional scrutiny during the PR review process, which is unfortunate but important.

Use your best judgement and respect that other people, across all timezones, rely on this repository remaining a productive working environment 24/7/365.

Updating the CI Environment

The continuous integration (CI) suite executes the same . docker/run-test command that developers are encouraged to run.

What happens if I need to change that environment? E.g. what if I need a new system library dependency installed?

  • Step 1 - Add the dependency to docker/Dockerfile.ubuntu
RUN apt-get update && apt-get install --yes\
  # ... snip ...
  my-new-lib-here
  • Step 2 - Build it
. docker/build-ubuntu
  • Step 3 - Test it out
. docker/run-test
  • Step 4 - Wait a minute! The CI environment is still using the old Dockerfile!

If your changes do not break the current environment, you can submit a separate Pull Request first, and once it is merged, the CI environment should be up-to-date for your code change Pull Request.

Otherwise, you will need to speak to an admin who can force merge your full changes after testing locally.

Building for Android

Note there is an article written on how to build Holochain for Android, read it here.

Contribute

Holochain is an open source project. We welcome all sorts of participation and are actively working on increasing surface area to accept it. Please see our contributing guidelines for our general practices and protocols on participating in the community.

App Spec Driven Development

In adding significant changes and new features to Holochain, we follow a specific test-driven development protocol:

  1. Start by creating a branch in this repository and modifying the example app in the app_spec directory to demonstrates an actual implementation of the use of the new feature, including tests that would pass if the feature were actually implemented.
  2. Create a pull request on that branch for the development team to talk about and discuss the suggested change. The PR triggers Continuous Integration tests which will initially fail.
  3. Do any development necessary in core or hdk crates of this repo to actually implement the feature demonstrated in app_spec
  4. Finally, when the feature is fully implemented, the CI tests should turn green and the branch can be merged.

In this way app_spec works as a living specification with example app to build against.

Some helpful links:

Current Throughput graph:

Throughput Graph

License

License: GPL v3

Copyright (C) 2018, Holochain Trust

This program is free software: you can redistribute it and/or modify it under the terms of the license p rovided in the LICENSE file (GPLv3). This program is distributed in the hope that it will be useful, bu t WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Note: We are considering other 'looser' licensing options (like MIT license) but at this stage are using GPL while we're getting the matter sorted out. See this article for some of our thinking on licensing for distributed application frameworks.