Add developer documentation

.github/workflows/pages.yml

@@ -0,0 +1,65 @@
+name: pages-haura-getting-started
+
+on:
+  workflow_dispatch:
+  push:
+    branches: [main]
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+concurrency:
+  group: "pages"
+  cancel-in-progress: true
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+    - name: Checkout
+      uses: actions/checkout@v3
+      with:
+        ref: ${{ github.event.pull_request.head.sha }}
+    - name: Configure Pages
+      uses: actions/configure-pages@v2
+    - name: Binary Cache
+      uses: actions/cache@v3
+      with:
+        path: |
+          bin
+        key: ${{ runner.os }}-cargo-mdbook
+    - name: Install mdbook
+      run: |
+        mkdir bin
+        curl -sSL https://github.com/rust-lang/mdBook/releases/download/v0.4.21/mdbook-v0.4.21-x86_64-unknown-linux-gnu.tar.gz | tar -xz --directory=bin
+        curl -sSL https://github.com/dylanowen/mdbook-graphviz/releases/download/v0.1.5/mdbook-graphviz_v0.1.5_x86_64-unknown-linux-musl.zip > mdviz.zip
+        unzip mdviz.zip -d bin
+    - name: Install graphviz
+      run: |
+        sudo apt update
+        sudo apt install -y graphviz
+    - name: Build book
+      run: |
+        PATH=$PATH:$PWD/bin
+        cd docs
+        ../bin/mdbook build
+    - name: Upload Book Artifact
+      uses: actions/upload-pages-artifact@v1
+      with:
+        path: ./docs/book
+  deploy:
+    runs-on: ubuntu-latest
+    needs: build
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+    - name: Checkout
+      uses: actions/checkout@v3
+      with:
+        ref: ${{ github.event.pull_request.head.sha }}
+    - name: Deploy Pages
+      id: deployment
+      uses: actions/deploy-pages@v1

.gitignore

@@ -3,3 +3,4 @@
 **/*.swp
 heaptrack*
 Cargo.lock
+docs/book

README.md

@@ -14,6 +14,21 @@ cd tests
 cargo test
 ```
 
+## Documentation
+
+You can find an in-depth documentation and developer guide under
+https://julea-io.github.io/haura or you may build it yourself locally.  For
+building the documentation [`mdbook`](https://rust-lang.github.io/mdBook/) is
+required. You can find install directions in their documentation under
+https://rust-lang.github.io/mdBook/.
+
+Also you'll require [mdbook-grapviz](https://lib.rs/crates/mdbook-graphviz) to
+render graphs within the documentation. This crate also needs graphviz to be
+installed on your system.
+
+The documentation is automatically built and published on GitHub Pages - the workflow for
+which you can find under `.github/workflows/pages.yml`.
+
 ### julea-sys
 
 `julea-sys` generates limited Rust bindings from the JULEA headers, by using rust-bindgen, which uses libclang.

docs/.gitignore

@@ -0,0 +1 @@
+book

docs/book.toml

@@ -0,0 +1,14 @@
+[book]
+authors = ["Johannes Wünsche"]
+language = "en"
+multilingual = false
+src = "src"
+title = "Haura Developer Guide"
+
+[output.html]
+site-url = "/haura/"
+
+[preprocessor.graphviz]
+command = "mdbook-graphviz"
+output-to-file = false
+

docs/src/SUMMARY.md

@@ -0,0 +1,17 @@
+# Haura Developer Guide
+
+- [Haura](./haura.md)
+- [Build](./build.md)
+- [Structure](./structure.md)
+  - [betree\_storage\_stack](./betree.md)
+    - [Build & Test](./betree/build.md)
+    - [Design Overview](./betree/design.md)
+    - [Known Bugs](./betree/bugs.md)
+  - [bectl](./bectl.md)
+    - [Build & Test](./bectl/build.md)
+    - [Basic Usage](./bectl/usage.md)
+  - [julea-betree](./julea-betree.md)
+    - [Build](./julea-betree/build.md)
+    - [Usage in JULEA](./julea-betree/usage.md)
+  - [julea-sys](./julea-sys.md)
+    - [Build](./julea-sys/build.md)

docs/src/bectl.md

@@ -0,0 +1,7 @@
+# bectl
+
+This application allows for usage with a CLI albeit suboptimally as the
+complete stack has to be reinitialized on each access. It is best fit for
+debugging at the moment and demonstration purposes.
+
+We shortly demonstrate how to build it and how to configure and use it.

docs/src/bectl/build.md

@@ -0,0 +1,29 @@
+# Build & Test
+
+
+Building `bectl` is relatively simple but does require all dependencies of
+`betree_storage_stack` to be available. If you have not done this yet refer to
+the [Building chapter](../build.md).
+
+Given all prerequisites are fulfilled simply run
+
+```sh
+$ cargo build
+```
+
+from the `bectl` directory to build `bectl` on its own.
+
+To avoid specifiyng the location of the binary or running `bectl` with `cargo
+run` on each invocation you can also install the app via `cargo` to your local
+user.
+
+```sh
+$ cargo install --path .
+```
+
+--- 
+
+There are not yet any tests provided for the `bectl` as the functionality is a
+rather simple mapping to `betree_storage_stack` functions. If we want to expand
+this in the future we might want to ensure that `bectl` behaves correctly
+internally too.

docs/src/bectl/usage.md

@@ -0,0 +1,107 @@
+# Basic Usage
+
+To use the storage stack via `bectl` you will first need to define a valid
+configuration. This can be done either by modifying a small example like shown
+below or by creating a valid configuraiton with the `betree_storage_stack` crate
+and calling `write_config_json` on the created `Database` object.
+
+##### Example Configuration
+```json
+{
+  "storage": {
+    "tiers": [
+      {
+        "top_level_vdevs": [
+          {
+            "path": "/home/user/.cache/haura/cache.disk",
+            "direct": true
+          }
+        ],
+        "preferred_access_type": "Unknown"
+      }
+    ],
+    "queue_depth_factor": 20,
+    "thread_pool_size": null,
+    "thread_pool_pinned": false
+  },
+  "alloc_strategy": [
+    [
+      0
+    ],
+    [
+      1
+    ],
+    [
+      2
+    ],
+    [
+      3
+    ]
+  ],
+  "default_storage_class": 0,
+  "compression": "None",
+  "cache_size": 4294967296,
+  "access_mode": "OpenOrCreate",
+  "sync_interval_ms": null,
+  "migration_policy": null,
+  "metrics": null
+}
+```
+
+Store this configuration in a convenient place for example, if defined, under
+`$XDG_CONFIG_HOME/haura.json` or alternatively `$HOME/.config/haura.json`. To
+avoid specifying this path on each access, store the location of your
+configuration in your environment as `$BETREE_CONFIG`. For example as so:
+
+
+```sh
+# excerpt from .your_favorite_shellenv
+
+export BETREE_CONFIG=$HOME/.config/haura.json
+```
+
+We use in our example `$XDG_CACHE_HOME/haura/cache.disk` as the storage file, this
+integrates nicely with most common setups and indicates that this data is not
+essential. If `XDG_CACHE_HOME` is not set in your system you can use
+`$HOME/.cache/haura/cache.disk` instead. Create the directories if they do not
+exists.
+
+```sh
+$ mkdir -p $HOME/.cache/haura
+$ truncate -s 16G $HOME/.cache/haura/cache.disk
+```
+
+> EDITOR NOTE: We may want to specify this as a `yaml` or `toml` in the future to ease the
+> configuration and actually be able to write this by hand besides the zfs-like
+> configuration string.
+
+## Usage
+
+Once you have configured everything you are ready to use `bectl`. We show you
+here the very basics on how to start using the CLI, you may find further in the
+CLI itself from various help pages.
+
+### Initialization
+
+Before using you'll need to initialize your storage.
+
+```sh
+$ bectl db init
+```
+
+### Write Data
+
+To write some value *baz* in the dataset *foo* in key *bar*.
+
+```sh
+bectl kv foo put bar baz
+```
+
+### Read Data
+
+To read some data from dataset *foo* in key *bar*.
+
+```sh
+bectl kv foo get bar
+```
+

docs/src/betree.md

@@ -0,0 +1,6 @@
+# B-epsilon Tree Storage Stack
+
+This crate is the main event of the repository as it contains all algorithmic
+logic required in the actual implementation of the B-epsilon tree storage.  We
+give you a quick introduction into the structure of the repository here and give
+you starting points where to start your implementation.