diff --git a/README.md b/README.md
index cce20514..02f5913e 100644
--- a/README.md
+++ b/README.md
@@ -2,14 +2,16 @@
## Introduction
-Leaf is a Machine Intelligence Framework engineered by software developers, not
-scientists. It was inspired by the brilliant people behind TensorFlow, Torch,
-Caffe, Rust and numerous research papers and brings modularity, performance and
-portability to deep learning. Leaf is lean and tries to introduce minimal
+Leaf is a open Machine Learning Framework for hackers to build classical, deep
+or hybrid machine learning applications. It was inspired by the brilliant people
+behind TensorFlow, Torch, Caffe, Rust and numerous research papers and brings
+modularity, performance and portability to deep learning.
+
+Leaf has one of the simplest APIs, is lean and tries to introduce minimal
technical debt to your stack.
-Leaf is a few months old, but thanks to its architecture and Rust, it is already one of
-the fastest Machine Intelligence Frameworks in the world.
+Leaf is a few months old, but thanks to its architecture and Rust, it is already
+one of the fastest Machine Intelligence Frameworks available.
@@ -17,13 +19,12 @@ the fastest Machine Intelligence Frameworks in the world.
> See more Deep Neural Networks benchmarks on [Deep Learning Benchmarks][deep-learning-benchmarks-website].
-Leaf is portable. Run it on CPUs, GPUs, FPGAs on machines with an OS or on
+Leaf is portable. Run it on CPUs, GPUs, and FPGAs, on machines with an OS, or on
machines without one. Run it with OpenCL or CUDA. Credit goes to
[Collenchyma][collenchyma] and Rust.
Leaf is part of the [Autumn][autumn] Machine Intelligence Platform, which is
-working on making AI algorithms 100x more computational efficient. It seeks to bring
-real-time, offline AI to smartphones and embedded devices.
+working on making AI algorithms 100x more computational efficient.
We see Leaf as the core of constructing high-performance machine intelligence
applications. Leaf's design makes it easy to publish independent modules to make
@@ -31,15 +32,10 @@ e.g. deep reinforcement learning, visualization and monitoring, network
distribution, [automated preprocessing][cuticula] or scaleable production
deployment easily accessible for everyone.
-For more info, refer to
-* the [Leaf examples][leaf-examples],
-* the [Leaf Documentation][documentation],
-* the [Autumn Website][autumn] or
-* the [Q&A](#qa)
-
[caffe]: https://github.com/BVLC/caffe
[rust]: https://www.rust-lang.org/
[autumn]: http://autumnai.com
+[leaf-book]: http://autumnai.com/leaf/book
[tensorflow]: https://github.com/tensorflow/tensorflow
[benchmarks]: #benchmarks
[leaf-examples]: #examples
@@ -52,22 +48,42 @@ For more info, refer to
## Getting Started
-If you are new to Rust you can install it as detailed [here][rust_download].
-We also recommend taking a look at the [official Getting Started Guide][rust_getting_started].
+### Documentation
+
+To learn how to build classical, deep or hybrid machine learning applications with Leaf, check out the [Leaf - Machine Learning for Hackers][leaf-book] book.
+
+For additional information see the [Rust API Documentation][documentation] or the [Autumn Website][autumn].
+
+Or start by running the **Leaf examples**.
-If you're using Cargo, just add Leaf to your `Cargo.toml`:
+We are providing a [Leaf examples repository][leaf-examples], where we and
+others publish executable machine learning models build with Leaf. It features
+a CLI for easy usage and has a detailed guide in the [project
+README.md][leaf-examples].
+
+Leaf comes with an examples directory as well, which features popular neural
+networks (e.g. Alexnet, Overfeat, VGG). To run them on your machine, just follow
+the install guide, clone this repoistory and then run
+
+```bash
+# The examples currently require CUDA support.
+cargo run --release --no-default-features --features cuda --example benchmarks alexnet
+```
+
+[leaf-examples]: https://github.com/autumnai/leaf-examples
+
+### Installation
+
+> Leaf is build in [Rust][rust]. If you are new to Rust you can install Rust as detailed [here][rust_download].
+We also recommend taking a look at the [official Rust - Getting Started Guide][rust_getting_started].
+
+To start building a machine learning application (Rust only for now. Wrappers are welcome) and you are using Cargo, just add Leaf to your `Cargo.toml`:
```toml
[dependencies]
leaf = "0.2.0"
```
-If you're using [Cargo Edit][cargo-edit], you can
-call:
-
-```bash
-cargo add leaf
-```
[rust_download]: https://www.rust-lang.org/downloads.html
[rust_getting_started]: https://doc.rust-lang.org/book/getting-started.html
[cargo-edit]: https://github.com/killercup/cargo-edit
@@ -88,24 +104,24 @@ opencl = ["leaf/opencl"]
> More information on the use of feature flags in Leaf can be found in [FEATURE-FLAGS.md](./FEATURE-FLAGS.md)
+### Contributing
-## Examples
+If you want to start hacking on Leaf (e.g.
+ [adding a new `Layer`](http://autumnai.com/leaf/book/create-new-layer.html))
+you should start with forking and cloning the repository.
-We are providing a [Leaf examples repository][leaf-examples], where we and
-others publish executable machine learning models build with Leaf. It features
-a CLI for easy usage and has a detailed guide in the [project
-README.md][leaf-examples].
+We have more instructions to help you get started in the [CONTRIBUTING.md][contributing].
-Leaf comes with an examples directory as well, which features popular neural
-networks (e.g. Alexnet, Overfeat, VGG). To run them on your machine, just follow
-the install guide, clone this repoistory and then run
+We also has a near real-time collaboration culture, which happens
+here on Github and on the [Leaf Gitter Channel][gitter-leaf].
-```bash
-# The examples currently require CUDA support.
-cargo run --release --no-default-features --features cuda --example benchmarks alexnet
-```
+> Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as below, without any additional terms or conditions.
-[leaf-examples]: https://github.com/autumnai/leaf-examples
+[contributing]: CONTRIBUTING.md
+[gitter-leaf]: https://gitter.im/autumnai/leaf
+[mj]: https://twitter.com/mjhirn
+[hobofan]: https://twitter.com/hobofan
+[irc]: https://chat.mibbit.com/?server=irc.mozilla.org&channel=%23rust-machine-learning
## Ecosystem / Extensions
@@ -120,7 +136,7 @@ and extensible as possible. More helpful crates you can use with Leaf:
## Support / Contact
-- With a bit of luck, you can find us online on the #rust-machine-learing IRC at irc.mozilla.org,
+- With a bit of luck, you can find us online on the #rust-machine-learning IRC at irc.mozilla.org,
- but we are always approachable on [Gitter/Leaf][gitter-leaf]
- For bugs and feature request, you can create a [Github issue][leaf-issue]
- For more private matters, send us email straight to our inbox: developers@autumnai.com
@@ -128,23 +144,6 @@ and extensible as possible. More helpful crates you can use with Leaf:
[leaf-issue]: https://github.com/autumnai/leaf/issues
-## Contributing
-
-Want to contribute? Awesome! We have [instructions to help you get started][contributing].
-
-Leaf has a near real-time collaboration culture, and it happens here on Github and
-on the [Leaf Gitter Channel][gitter-leaf].
-
-Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0
-license, shall be dual licensed as below, without any additional terms or
-conditions.
-
-[contributing]: CONTRIBUTING.md
-[gitter-leaf]: https://gitter.im/autumnai/leaf
-[mj]: https://twitter.com/mjhirn
-[hobofan]: https://twitter.com/hobofan
-[irc]: https://chat.mibbit.com/?server=irc.mozilla.org&channel=%23rust-machine-learning
-
## Changelog
You can find the release history at the [CHANGELOG.md][changelog]. We are using [Clog][clog], the Rust tool for auto-generating CHANGELOG files.
@@ -152,52 +151,6 @@ You can find the release history at the [CHANGELOG.md][changelog]. We are using
[changelog]: CHANGELOG.md
[Clog]: https://github.com/clog-tool/clog-cli
-## Q&A
-
-#### _Why Rust?_
-
-Hardware has just recently become strong enough to support real-world
-usage of machine intelligence e.g. super-human image recognition, self-driving
-cars, etc. To take advantage of the computational power of the underlying
-hardware, from GPUs to clusters, you need a low-level language that allows for
-control of memory. But to make machine intelligence widely accessible you want
-to have a high-level, comfortable abstraction over the underlying hardware.
-
-Rust allows us to cross this chasm.
-Rust promises performance like C/C++ but with safe memory-control. For now we
-can use C Rust wrappers for performant libraries. But in the future Rust
-rewritten libraries will have the advantage of zero-cost safe memory control,
-that will make large, parallel learning networks over CPUs and GPUs more
-feasible and more reliable to develop. The development of these future libraries
-is already under way e.g. [Glium][glium].
-
-On the usability side, Rust offers a trait-system that makes it easy for
-researchers and hobbyists alike to extend and work with Leaf as if it were
-written in a higher-level language such as Ruby, Python, or Java.
-
-#### _Who can use Leaf?_
-
-We develop Leaf under the MIT open source license, which, paired with the easy
-access and performance, makes Leaf a first-choice option for researchers and
-developers alike.
-
-#### _Why did you open source Leaf?_
-
-We believe strongly in machine intelligence and think that it will have a major
-impact on future innovations, products and our society. At Autumn, we experienced
-a lack of common and well engineered tools for machine learning and therefore
-started to create a modular toolbox for machine learning in Rust. We hope that,
-by making our work open source, we will speed up research and development of
-production-ready applications and make that work easier as well.
-
-#### _Who is Autumn?_
-
-Autumn is a startup working on automated decision making. Autumn was started by
-two developers, MJ and Max. The startup is located in Berlin and recently
-received a pre-seed investment from Axel Springer and Plug&Play.
-
-[glium]: https://github.com/tomaka/glium
-
## License
Licensed under either of
diff --git a/doc/book/_FontAwesome/css/font-awesome.css b/doc/book/_FontAwesome/css/font-awesome.css
new file mode 100644
index 00000000..ee4e9782
--- /dev/null
+++ b/doc/book/_FontAwesome/css/font-awesome.css
@@ -0,0 +1,4 @@
+/*!
+ * Font Awesome 4.4.0 by @davegandy - http://fontawesome.io - @fontawesome
+ * License - http://fontawesome.io/license (Font: SIL OFL 1.1, CSS: MIT License)
+ */@font-face{font-family:'FontAwesome';src:url('../fonts/fontawesome-webfont.eot?v=4.4.0');src:url('../fonts/fontawesome-webfont.eot?#iefix&v=4.4.0') format('embedded-opentype'),url('../fonts/fontawesome-webfont.woff2?v=4.4.0') format('woff2'),url('../fonts/fontawesome-webfont.woff?v=4.4.0') format('woff'),url('../fonts/fontawesome-webfont.ttf?v=4.4.0') format('truetype'),url('../fonts/fontawesome-webfont.svg?v=4.4.0#fontawesomeregular') format('svg');font-weight:normal;font-style:normal}.fa{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale}.fa-lg{font-size:1.33333333em;line-height:.75em;vertical-align:-15%}.fa-2x{font-size:2em}.fa-3x{font-size:3em}.fa-4x{font-size:4em}.fa-5x{font-size:5em}.fa-fw{width:1.28571429em;text-align:center}.fa-ul{padding-left:0;margin-left:2.14285714em;list-style-type:none}.fa-ul>li{position:relative}.fa-li{position:absolute;left:-2.14285714em;width:2.14285714em;top:.14285714em;text-align:center}.fa-li.fa-lg{left:-1.85714286em}.fa-border{padding:.2em .25em .15em;border:solid .08em #eee;border-radius:.1em}.fa-pull-left{float:left}.fa-pull-right{float:right}.fa.fa-pull-left{margin-right:.3em}.fa.fa-pull-right{margin-left:.3em}.pull-right{float:right}.pull-left{float:left}.fa.pull-left{margin-right:.3em}.fa.pull-right{margin-left:.3em}.fa-spin{-webkit-animation:fa-spin 2s infinite linear;animation:fa-spin 2s infinite linear}.fa-pulse{-webkit-animation:fa-spin 1s infinite steps(8);animation:fa-spin 1s infinite steps(8)}@-webkit-keyframes fa-spin{0%{-webkit-transform:rotate(0deg);transform:rotate(0deg)}100%{-webkit-transform:rotate(359deg);transform:rotate(359deg)}}@keyframes fa-spin{0%{-webkit-transform:rotate(0deg);transform:rotate(0deg)}100%{-webkit-transform:rotate(359deg);transform:rotate(359deg)}}.fa-rotate-90{filter:progid:DXImageTransform.Microsoft.BasicImage(rotation=1);-webkit-transform:rotate(90deg);-ms-transform:rotate(90deg);transform:rotate(90deg)}.fa-rotate-180{filter:progid:DXImageTransform.Microsoft.BasicImage(rotation=2);-webkit-transform:rotate(180deg);-ms-transform:rotate(180deg);transform:rotate(180deg)}.fa-rotate-270{filter:progid:DXImageTransform.Microsoft.BasicImage(rotation=3);-webkit-transform:rotate(270deg);-ms-transform:rotate(270deg);transform:rotate(270deg)}.fa-flip-horizontal{filter:progid:DXImageTransform.Microsoft.BasicImage(rotation=0, mirror=1);-webkit-transform:scale(-1, 1);-ms-transform:scale(-1, 1);transform:scale(-1, 1)}.fa-flip-vertical{filter:progid:DXImageTransform.Microsoft.BasicImage(rotation=2, mirror=1);-webkit-transform:scale(1, -1);-ms-transform:scale(1, -1);transform:scale(1, -1)}:root .fa-rotate-90,:root .fa-rotate-180,:root .fa-rotate-270,:root .fa-flip-horizontal,:root .fa-flip-vertical{filter:none}.fa-stack{position:relative;display:inline-block;width:2em;height:2em;line-height:2em;vertical-align:middle}.fa-stack-1x,.fa-stack-2x{position:absolute;left:0;width:100%;text-align:center}.fa-stack-1x{line-height:inherit}.fa-stack-2x{font-size:2em}.fa-inverse{color:#fff}.fa-glass:before{content:"\f000"}.fa-music:before{content:"\f001"}.fa-search:before{content:"\f002"}.fa-envelope-o:before{content:"\f003"}.fa-heart:before{content:"\f004"}.fa-star:before{content:"\f005"}.fa-star-o:before{content:"\f006"}.fa-user:before{content:"\f007"}.fa-film:before{content:"\f008"}.fa-th-large:before{content:"\f009"}.fa-th:before{content:"\f00a"}.fa-th-list:before{content:"\f00b"}.fa-check:before{content:"\f00c"}.fa-remove:before,.fa-close:before,.fa-times:before{content:"\f00d"}.fa-search-plus:before{content:"\f00e"}.fa-search-minus:before{content:"\f010"}.fa-power-off:before{content:"\f011"}.fa-signal:before{content:"\f012"}.fa-gear:before,.fa-cog:before{content:"\f013"}.fa-trash-o:before{content:"\f014"}.fa-home:before{content:"\f015"}.fa-file-o:before{content:"\f016"}.fa-clock-o:before{content:"\f017"}.fa-road:before{content:"\f018"}.fa-download:before{content:"\f019"}.fa-arrow-circle-o-down:before{content:"\f01a"}.fa-arrow-circle-o-up:before{content:"\f01b"}.fa-inbox:before{content:"\f01c"}.fa-play-circle-o:before{content:"\f01d"}.fa-rotate-right:before,.fa-repeat:before{content:"\f01e"}.fa-refresh:before{content:"\f021"}.fa-list-alt:before{content:"\f022"}.fa-lock:before{content:"\f023"}.fa-flag:before{content:"\f024"}.fa-headphones:before{content:"\f025"}.fa-volume-off:before{content:"\f026"}.fa-volume-down:before{content:"\f027"}.fa-volume-up:before{content:"\f028"}.fa-qrcode:before{content:"\f029"}.fa-barcode:before{content:"\f02a"}.fa-tag:before{content:"\f02b"}.fa-tags:before{content:"\f02c"}.fa-book:before{content:"\f02d"}.fa-bookmark:before{content:"\f02e"}.fa-print:before{content:"\f02f"}.fa-camera:before{content:"\f030"}.fa-font:before{content:"\f031"}.fa-bold:before{content:"\f032"}.fa-italic:before{content:"\f033"}.fa-text-height:before{content:"\f034"}.fa-text-width:before{content:"\f035"}.fa-align-left:before{content:"\f036"}.fa-align-center:before{content:"\f037"}.fa-align-right:before{content:"\f038"}.fa-align-justify:before{content:"\f039"}.fa-list:before{content:"\f03a"}.fa-dedent:before,.fa-outdent:before{content:"\f03b"}.fa-indent:before{content:"\f03c"}.fa-video-camera:before{content:"\f03d"}.fa-photo:before,.fa-image:before,.fa-picture-o:before{content:"\f03e"}.fa-pencil:before{content:"\f040"}.fa-map-marker:before{content:"\f041"}.fa-adjust:before{content:"\f042"}.fa-tint:before{content:"\f043"}.fa-edit:before,.fa-pencil-square-o:before{content:"\f044"}.fa-share-square-o:before{content:"\f045"}.fa-check-square-o:before{content:"\f046"}.fa-arrows:before{content:"\f047"}.fa-step-backward:before{content:"\f048"}.fa-fast-backward:before{content:"\f049"}.fa-backward:before{content:"\f04a"}.fa-play:before{content:"\f04b"}.fa-pause:before{content:"\f04c"}.fa-stop:before{content:"\f04d"}.fa-forward:before{content:"\f04e"}.fa-fast-forward:before{content:"\f050"}.fa-step-forward:before{content:"\f051"}.fa-eject:before{content:"\f052"}.fa-chevron-left:before{content:"\f053"}.fa-chevron-right:before{content:"\f054"}.fa-plus-circle:before{content:"\f055"}.fa-minus-circle:before{content:"\f056"}.fa-times-circle:before{content:"\f057"}.fa-check-circle:before{content:"\f058"}.fa-question-circle:before{content:"\f059"}.fa-info-circle:before{content:"\f05a"}.fa-crosshairs:before{content:"\f05b"}.fa-times-circle-o:before{content:"\f05c"}.fa-check-circle-o:before{content:"\f05d"}.fa-ban:before{content:"\f05e"}.fa-arrow-left:before{content:"\f060"}.fa-arrow-right:before{content:"\f061"}.fa-arrow-up:before{content:"\f062"}.fa-arrow-down:before{content:"\f063"}.fa-mail-forward:before,.fa-share:before{content:"\f064"}.fa-expand:before{content:"\f065"}.fa-compress:before{content:"\f066"}.fa-plus:before{content:"\f067"}.fa-minus:before{content:"\f068"}.fa-asterisk:before{content:"\f069"}.fa-exclamation-circle:before{content:"\f06a"}.fa-gift:before{content:"\f06b"}.fa-leaf:before{content:"\f06c"}.fa-fire:before{content:"\f06d"}.fa-eye:before{content:"\f06e"}.fa-eye-slash:before{content:"\f070"}.fa-warning:before,.fa-exclamation-triangle:before{content:"\f071"}.fa-plane:before{content:"\f072"}.fa-calendar:before{content:"\f073"}.fa-random:before{content:"\f074"}.fa-comment:before{content:"\f075"}.fa-magnet:before{content:"\f076"}.fa-chevron-up:before{content:"\f077"}.fa-chevron-down:before{content:"\f078"}.fa-retweet:before{content:"\f079"}.fa-shopping-cart:before{content:"\f07a"}.fa-folder:before{content:"\f07b"}.fa-folder-open:before{content:"\f07c"}.fa-arrows-v:before{content:"\f07d"}.fa-arrows-h:before{content:"\f07e"}.fa-bar-chart-o:before,.fa-bar-chart:before{content:"\f080"}.fa-twitter-square:before{content:"\f081"}.fa-facebook-square:before{content:"\f082"}.fa-camera-retro:before{content:"\f083"}.fa-key:before{content:"\f084"}.fa-gears:before,.fa-cogs:before{content:"\f085"}.fa-comments:before{content:"\f086"}.fa-thumbs-o-up:before{content:"\f087"}.fa-thumbs-o-down:before{content:"\f088"}.fa-star-half:before{content:"\f089"}.fa-heart-o:before{content:"\f08a"}.fa-sign-out:before{content:"\f08b"}.fa-linkedin-square:before{content:"\f08c"}.fa-thumb-tack:before{content:"\f08d"}.fa-external-link:before{content:"\f08e"}.fa-sign-in:before{content:"\f090"}.fa-trophy:before{content:"\f091"}.fa-github-square:before{content:"\f092"}.fa-upload:before{content:"\f093"}.fa-lemon-o:before{content:"\f094"}.fa-phone:before{content:"\f095"}.fa-square-o:before{content:"\f096"}.fa-bookmark-o:before{content:"\f097"}.fa-phone-square:before{content:"\f098"}.fa-twitter:before{content:"\f099"}.fa-facebook-f:before,.fa-facebook:before{content:"\f09a"}.fa-github:before{content:"\f09b"}.fa-unlock:before{content:"\f09c"}.fa-credit-card:before{content:"\f09d"}.fa-feed:before,.fa-rss:before{content:"\f09e"}.fa-hdd-o:before{content:"\f0a0"}.fa-bullhorn:before{content:"\f0a1"}.fa-bell:before{content:"\f0f3"}.fa-certificate:before{content:"\f0a3"}.fa-hand-o-right:before{content:"\f0a4"}.fa-hand-o-left:before{content:"\f0a5"}.fa-hand-o-up:before{content:"\f0a6"}.fa-hand-o-down:before{content:"\f0a7"}.fa-arrow-circle-left:before{content:"\f0a8"}.fa-arrow-circle-right:before{content:"\f0a9"}.fa-arrow-circle-up:before{content:"\f0aa"}.fa-arrow-circle-down:before{content:"\f0ab"}.fa-globe:before{content:"\f0ac"}.fa-wrench:before{content:"\f0ad"}.fa-tasks:before{content:"\f0ae"}.fa-filter:before{content:"\f0b0"}.fa-briefcase:before{content:"\f0b1"}.fa-arrows-alt:before{content:"\f0b2"}.fa-group:before,.fa-users:before{content:"\f0c0"}.fa-chain:before,.fa-link:before{content:"\f0c1"}.fa-cloud:before{content:"\f0c2"}.fa-flask:before{content:"\f0c3"}.fa-cut:before,.fa-scissors:before{content:"\f0c4"}.fa-copy:before,.fa-files-o:before{content:"\f0c5"}.fa-paperclip:before{content:"\f0c6"}.fa-save:before,.fa-floppy-o:before{content:"\f0c7"}.fa-square:before{content:"\f0c8"}.fa-navicon:before,.fa-reorder:before,.fa-bars:before{content:"\f0c9"}.fa-list-ul:before{content:"\f0ca"}.fa-list-ol:before{content:"\f0cb"}.fa-strikethrough:before{content:"\f0cc"}.fa-underline:before{content:"\f0cd"}.fa-table:before{content:"\f0ce"}.fa-magic:before{content:"\f0d0"}.fa-truck:before{content:"\f0d1"}.fa-pinterest:before{content:"\f0d2"}.fa-pinterest-square:before{content:"\f0d3"}.fa-google-plus-square:before{content:"\f0d4"}.fa-google-plus:before{content:"\f0d5"}.fa-money:before{content:"\f0d6"}.fa-caret-down:before{content:"\f0d7"}.fa-caret-up:before{content:"\f0d8"}.fa-caret-left:before{content:"\f0d9"}.fa-caret-right:before{content:"\f0da"}.fa-columns:before{content:"\f0db"}.fa-unsorted:before,.fa-sort:before{content:"\f0dc"}.fa-sort-down:before,.fa-sort-desc:before{content:"\f0dd"}.fa-sort-up:before,.fa-sort-asc:before{content:"\f0de"}.fa-envelope:before{content:"\f0e0"}.fa-linkedin:before{content:"\f0e1"}.fa-rotate-left:before,.fa-undo:before{content:"\f0e2"}.fa-legal:before,.fa-gavel:before{content:"\f0e3"}.fa-dashboard:before,.fa-tachometer:before{content:"\f0e4"}.fa-comment-o:before{content:"\f0e5"}.fa-comments-o:before{content:"\f0e6"}.fa-flash:before,.fa-bolt:before{content:"\f0e7"}.fa-sitemap:before{content:"\f0e8"}.fa-umbrella:before{content:"\f0e9"}.fa-paste:before,.fa-clipboard:before{content:"\f0ea"}.fa-lightbulb-o:before{content:"\f0eb"}.fa-exchange:before{content:"\f0ec"}.fa-cloud-download:before{content:"\f0ed"}.fa-cloud-upload:before{content:"\f0ee"}.fa-user-md:before{content:"\f0f0"}.fa-stethoscope:before{content:"\f0f1"}.fa-suitcase:before{content:"\f0f2"}.fa-bell-o:before{content:"\f0a2"}.fa-coffee:before{content:"\f0f4"}.fa-cutlery:before{content:"\f0f5"}.fa-file-text-o:before{content:"\f0f6"}.fa-building-o:before{content:"\f0f7"}.fa-hospital-o:before{content:"\f0f8"}.fa-ambulance:before{content:"\f0f9"}.fa-medkit:before{content:"\f0fa"}.fa-fighter-jet:before{content:"\f0fb"}.fa-beer:before{content:"\f0fc"}.fa-h-square:before{content:"\f0fd"}.fa-plus-square:before{content:"\f0fe"}.fa-angle-double-left:before{content:"\f100"}.fa-angle-double-right:before{content:"\f101"}.fa-angle-double-up:before{content:"\f102"}.fa-angle-double-down:before{content:"\f103"}.fa-angle-left:before{content:"\f104"}.fa-angle-right:before{content:"\f105"}.fa-angle-up:before{content:"\f106"}.fa-angle-down:before{content:"\f107"}.fa-desktop:before{content:"\f108"}.fa-laptop:before{content:"\f109"}.fa-tablet:before{content:"\f10a"}.fa-mobile-phone:before,.fa-mobile:before{content:"\f10b"}.fa-circle-o:before{content:"\f10c"}.fa-quote-left:before{content:"\f10d"}.fa-quote-right:before{content:"\f10e"}.fa-spinner:before{content:"\f110"}.fa-circle:before{content:"\f111"}.fa-mail-reply:before,.fa-reply:before{content:"\f112"}.fa-github-alt:before{content:"\f113"}.fa-folder-o:before{content:"\f114"}.fa-folder-open-o:before{content:"\f115"}.fa-smile-o:before{content:"\f118"}.fa-frown-o:before{content:"\f119"}.fa-meh-o:before{content:"\f11a"}.fa-gamepad:before{content:"\f11b"}.fa-keyboard-o:before{content:"\f11c"}.fa-flag-o:before{content:"\f11d"}.fa-flag-checkered:before{content:"\f11e"}.fa-terminal:before{content:"\f120"}.fa-code:before{content:"\f121"}.fa-mail-reply-all:before,.fa-reply-all:before{content:"\f122"}.fa-star-half-empty:before,.fa-star-half-full:before,.fa-star-half-o:before{content:"\f123"}.fa-location-arrow:before{content:"\f124"}.fa-crop:before{content:"\f125"}.fa-code-fork:before{content:"\f126"}.fa-unlink:before,.fa-chain-broken:before{content:"\f127"}.fa-question:before{content:"\f128"}.fa-info:before{content:"\f129"}.fa-exclamation:before{content:"\f12a"}.fa-superscript:before{content:"\f12b"}.fa-subscript:before{content:"\f12c"}.fa-eraser:before{content:"\f12d"}.fa-puzzle-piece:before{content:"\f12e"}.fa-microphone:before{content:"\f130"}.fa-microphone-slash:before{content:"\f131"}.fa-shield:before{content:"\f132"}.fa-calendar-o:before{content:"\f133"}.fa-fire-extinguisher:before{content:"\f134"}.fa-rocket:before{content:"\f135"}.fa-maxcdn:before{content:"\f136"}.fa-chevron-circle-left:before{content:"\f137"}.fa-chevron-circle-right:before{content:"\f138"}.fa-chevron-circle-up:before{content:"\f139"}.fa-chevron-circle-down:before{content:"\f13a"}.fa-html5:before{content:"\f13b"}.fa-css3:before{content:"\f13c"}.fa-anchor:before{content:"\f13d"}.fa-unlock-alt:before{content:"\f13e"}.fa-bullseye:before{content:"\f140"}.fa-ellipsis-h:before{content:"\f141"}.fa-ellipsis-v:before{content:"\f142"}.fa-rss-square:before{content:"\f143"}.fa-play-circle:before{content:"\f144"}.fa-ticket:before{content:"\f145"}.fa-minus-square:before{content:"\f146"}.fa-minus-square-o:before{content:"\f147"}.fa-level-up:before{content:"\f148"}.fa-level-down:before{content:"\f149"}.fa-check-square:before{content:"\f14a"}.fa-pencil-square:before{content:"\f14b"}.fa-external-link-square:before{content:"\f14c"}.fa-share-square:before{content:"\f14d"}.fa-compass:before{content:"\f14e"}.fa-toggle-down:before,.fa-caret-square-o-down:before{content:"\f150"}.fa-toggle-up:before,.fa-caret-square-o-up:before{content:"\f151"}.fa-toggle-right:before,.fa-caret-square-o-right:before{content:"\f152"}.fa-euro:before,.fa-eur:before{content:"\f153"}.fa-gbp:before{content:"\f154"}.fa-dollar:before,.fa-usd:before{content:"\f155"}.fa-rupee:before,.fa-inr:before{content:"\f156"}.fa-cny:before,.fa-rmb:before,.fa-yen:before,.fa-jpy:before{content:"\f157"}.fa-ruble:before,.fa-rouble:before,.fa-rub:before{content:"\f158"}.fa-won:before,.fa-krw:before{content:"\f159"}.fa-bitcoin:before,.fa-btc:before{content:"\f15a"}.fa-file:before{content:"\f15b"}.fa-file-text:before{content:"\f15c"}.fa-sort-alpha-asc:before{content:"\f15d"}.fa-sort-alpha-desc:before{content:"\f15e"}.fa-sort-amount-asc:before{content:"\f160"}.fa-sort-amount-desc:before{content:"\f161"}.fa-sort-numeric-asc:before{content:"\f162"}.fa-sort-numeric-desc:before{content:"\f163"}.fa-thumbs-up:before{content:"\f164"}.fa-thumbs-down:before{content:"\f165"}.fa-youtube-square:before{content:"\f166"}.fa-youtube:before{content:"\f167"}.fa-xing:before{content:"\f168"}.fa-xing-square:before{content:"\f169"}.fa-youtube-play:before{content:"\f16a"}.fa-dropbox:before{content:"\f16b"}.fa-stack-overflow:before{content:"\f16c"}.fa-instagram:before{content:"\f16d"}.fa-flickr:before{content:"\f16e"}.fa-adn:before{content:"\f170"}.fa-bitbucket:before{content:"\f171"}.fa-bitbucket-square:before{content:"\f172"}.fa-tumblr:before{content:"\f173"}.fa-tumblr-square:before{content:"\f174"}.fa-long-arrow-down:before{content:"\f175"}.fa-long-arrow-up:before{content:"\f176"}.fa-long-arrow-left:before{content:"\f177"}.fa-long-arrow-right:before{content:"\f178"}.fa-apple:before{content:"\f179"}.fa-windows:before{content:"\f17a"}.fa-android:before{content:"\f17b"}.fa-linux:before{content:"\f17c"}.fa-dribbble:before{content:"\f17d"}.fa-skype:before{content:"\f17e"}.fa-foursquare:before{content:"\f180"}.fa-trello:before{content:"\f181"}.fa-female:before{content:"\f182"}.fa-male:before{content:"\f183"}.fa-gittip:before,.fa-gratipay:before{content:"\f184"}.fa-sun-o:before{content:"\f185"}.fa-moon-o:before{content:"\f186"}.fa-archive:before{content:"\f187"}.fa-bug:before{content:"\f188"}.fa-vk:before{content:"\f189"}.fa-weibo:before{content:"\f18a"}.fa-renren:before{content:"\f18b"}.fa-pagelines:before{content:"\f18c"}.fa-stack-exchange:before{content:"\f18d"}.fa-arrow-circle-o-right:before{content:"\f18e"}.fa-arrow-circle-o-left:before{content:"\f190"}.fa-toggle-left:before,.fa-caret-square-o-left:before{content:"\f191"}.fa-dot-circle-o:before{content:"\f192"}.fa-wheelchair:before{content:"\f193"}.fa-vimeo-square:before{content:"\f194"}.fa-turkish-lira:before,.fa-try:before{content:"\f195"}.fa-plus-square-o:before{content:"\f196"}.fa-space-shuttle:before{content:"\f197"}.fa-slack:before{content:"\f198"}.fa-envelope-square:before{content:"\f199"}.fa-wordpress:before{content:"\f19a"}.fa-openid:before{content:"\f19b"}.fa-institution:before,.fa-bank:before,.fa-university:before{content:"\f19c"}.fa-mortar-board:before,.fa-graduation-cap:before{content:"\f19d"}.fa-yahoo:before{content:"\f19e"}.fa-google:before{content:"\f1a0"}.fa-reddit:before{content:"\f1a1"}.fa-reddit-square:before{content:"\f1a2"}.fa-stumbleupon-circle:before{content:"\f1a3"}.fa-stumbleupon:before{content:"\f1a4"}.fa-delicious:before{content:"\f1a5"}.fa-digg:before{content:"\f1a6"}.fa-pied-piper:before{content:"\f1a7"}.fa-pied-piper-alt:before{content:"\f1a8"}.fa-drupal:before{content:"\f1a9"}.fa-joomla:before{content:"\f1aa"}.fa-language:before{content:"\f1ab"}.fa-fax:before{content:"\f1ac"}.fa-building:before{content:"\f1ad"}.fa-child:before{content:"\f1ae"}.fa-paw:before{content:"\f1b0"}.fa-spoon:before{content:"\f1b1"}.fa-cube:before{content:"\f1b2"}.fa-cubes:before{content:"\f1b3"}.fa-behance:before{content:"\f1b4"}.fa-behance-square:before{content:"\f1b5"}.fa-steam:before{content:"\f1b6"}.fa-steam-square:before{content:"\f1b7"}.fa-recycle:before{content:"\f1b8"}.fa-automobile:before,.fa-car:before{content:"\f1b9"}.fa-cab:before,.fa-taxi:before{content:"\f1ba"}.fa-tree:before{content:"\f1bb"}.fa-spotify:before{content:"\f1bc"}.fa-deviantart:before{content:"\f1bd"}.fa-soundcloud:before{content:"\f1be"}.fa-database:before{content:"\f1c0"}.fa-file-pdf-o:before{content:"\f1c1"}.fa-file-word-o:before{content:"\f1c2"}.fa-file-excel-o:before{content:"\f1c3"}.fa-file-powerpoint-o:before{content:"\f1c4"}.fa-file-photo-o:before,.fa-file-picture-o:before,.fa-file-image-o:before{content:"\f1c5"}.fa-file-zip-o:before,.fa-file-archive-o:before{content:"\f1c6"}.fa-file-sound-o:before,.fa-file-audio-o:before{content:"\f1c7"}.fa-file-movie-o:before,.fa-file-video-o:before{content:"\f1c8"}.fa-file-code-o:before{content:"\f1c9"}.fa-vine:before{content:"\f1ca"}.fa-codepen:before{content:"\f1cb"}.fa-jsfiddle:before{content:"\f1cc"}.fa-life-bouy:before,.fa-life-buoy:before,.fa-life-saver:before,.fa-support:before,.fa-life-ring:before{content:"\f1cd"}.fa-circle-o-notch:before{content:"\f1ce"}.fa-ra:before,.fa-rebel:before{content:"\f1d0"}.fa-ge:before,.fa-empire:before{content:"\f1d1"}.fa-git-square:before{content:"\f1d2"}.fa-git:before{content:"\f1d3"}.fa-y-combinator-square:before,.fa-yc-square:before,.fa-hacker-news:before{content:"\f1d4"}.fa-tencent-weibo:before{content:"\f1d5"}.fa-qq:before{content:"\f1d6"}.fa-wechat:before,.fa-weixin:before{content:"\f1d7"}.fa-send:before,.fa-paper-plane:before{content:"\f1d8"}.fa-send-o:before,.fa-paper-plane-o:before{content:"\f1d9"}.fa-history:before{content:"\f1da"}.fa-circle-thin:before{content:"\f1db"}.fa-header:before{content:"\f1dc"}.fa-paragraph:before{content:"\f1dd"}.fa-sliders:before{content:"\f1de"}.fa-share-alt:before{content:"\f1e0"}.fa-share-alt-square:before{content:"\f1e1"}.fa-bomb:before{content:"\f1e2"}.fa-soccer-ball-o:before,.fa-futbol-o:before{content:"\f1e3"}.fa-tty:before{content:"\f1e4"}.fa-binoculars:before{content:"\f1e5"}.fa-plug:before{content:"\f1e6"}.fa-slideshare:before{content:"\f1e7"}.fa-twitch:before{content:"\f1e8"}.fa-yelp:before{content:"\f1e9"}.fa-newspaper-o:before{content:"\f1ea"}.fa-wifi:before{content:"\f1eb"}.fa-calculator:before{content:"\f1ec"}.fa-paypal:before{content:"\f1ed"}.fa-google-wallet:before{content:"\f1ee"}.fa-cc-visa:before{content:"\f1f0"}.fa-cc-mastercard:before{content:"\f1f1"}.fa-cc-discover:before{content:"\f1f2"}.fa-cc-amex:before{content:"\f1f3"}.fa-cc-paypal:before{content:"\f1f4"}.fa-cc-stripe:before{content:"\f1f5"}.fa-bell-slash:before{content:"\f1f6"}.fa-bell-slash-o:before{content:"\f1f7"}.fa-trash:before{content:"\f1f8"}.fa-copyright:before{content:"\f1f9"}.fa-at:before{content:"\f1fa"}.fa-eyedropper:before{content:"\f1fb"}.fa-paint-brush:before{content:"\f1fc"}.fa-birthday-cake:before{content:"\f1fd"}.fa-area-chart:before{content:"\f1fe"}.fa-pie-chart:before{content:"\f200"}.fa-line-chart:before{content:"\f201"}.fa-lastfm:before{content:"\f202"}.fa-lastfm-square:before{content:"\f203"}.fa-toggle-off:before{content:"\f204"}.fa-toggle-on:before{content:"\f205"}.fa-bicycle:before{content:"\f206"}.fa-bus:before{content:"\f207"}.fa-ioxhost:before{content:"\f208"}.fa-angellist:before{content:"\f209"}.fa-cc:before{content:"\f20a"}.fa-shekel:before,.fa-sheqel:before,.fa-ils:before{content:"\f20b"}.fa-meanpath:before{content:"\f20c"}.fa-buysellads:before{content:"\f20d"}.fa-connectdevelop:before{content:"\f20e"}.fa-dashcube:before{content:"\f210"}.fa-forumbee:before{content:"\f211"}.fa-leanpub:before{content:"\f212"}.fa-sellsy:before{content:"\f213"}.fa-shirtsinbulk:before{content:"\f214"}.fa-simplybuilt:before{content:"\f215"}.fa-skyatlas:before{content:"\f216"}.fa-cart-plus:before{content:"\f217"}.fa-cart-arrow-down:before{content:"\f218"}.fa-diamond:before{content:"\f219"}.fa-ship:before{content:"\f21a"}.fa-user-secret:before{content:"\f21b"}.fa-motorcycle:before{content:"\f21c"}.fa-street-view:before{content:"\f21d"}.fa-heartbeat:before{content:"\f21e"}.fa-venus:before{content:"\f221"}.fa-mars:before{content:"\f222"}.fa-mercury:before{content:"\f223"}.fa-intersex:before,.fa-transgender:before{content:"\f224"}.fa-transgender-alt:before{content:"\f225"}.fa-venus-double:before{content:"\f226"}.fa-mars-double:before{content:"\f227"}.fa-venus-mars:before{content:"\f228"}.fa-mars-stroke:before{content:"\f229"}.fa-mars-stroke-v:before{content:"\f22a"}.fa-mars-stroke-h:before{content:"\f22b"}.fa-neuter:before{content:"\f22c"}.fa-genderless:before{content:"\f22d"}.fa-facebook-official:before{content:"\f230"}.fa-pinterest-p:before{content:"\f231"}.fa-whatsapp:before{content:"\f232"}.fa-server:before{content:"\f233"}.fa-user-plus:before{content:"\f234"}.fa-user-times:before{content:"\f235"}.fa-hotel:before,.fa-bed:before{content:"\f236"}.fa-viacoin:before{content:"\f237"}.fa-train:before{content:"\f238"}.fa-subway:before{content:"\f239"}.fa-medium:before{content:"\f23a"}.fa-yc:before,.fa-y-combinator:before{content:"\f23b"}.fa-optin-monster:before{content:"\f23c"}.fa-opencart:before{content:"\f23d"}.fa-expeditedssl:before{content:"\f23e"}.fa-battery-4:before,.fa-battery-full:before{content:"\f240"}.fa-battery-3:before,.fa-battery-three-quarters:before{content:"\f241"}.fa-battery-2:before,.fa-battery-half:before{content:"\f242"}.fa-battery-1:before,.fa-battery-quarter:before{content:"\f243"}.fa-battery-0:before,.fa-battery-empty:before{content:"\f244"}.fa-mouse-pointer:before{content:"\f245"}.fa-i-cursor:before{content:"\f246"}.fa-object-group:before{content:"\f247"}.fa-object-ungroup:before{content:"\f248"}.fa-sticky-note:before{content:"\f249"}.fa-sticky-note-o:before{content:"\f24a"}.fa-cc-jcb:before{content:"\f24b"}.fa-cc-diners-club:before{content:"\f24c"}.fa-clone:before{content:"\f24d"}.fa-balance-scale:before{content:"\f24e"}.fa-hourglass-o:before{content:"\f250"}.fa-hourglass-1:before,.fa-hourglass-start:before{content:"\f251"}.fa-hourglass-2:before,.fa-hourglass-half:before{content:"\f252"}.fa-hourglass-3:before,.fa-hourglass-end:before{content:"\f253"}.fa-hourglass:before{content:"\f254"}.fa-hand-grab-o:before,.fa-hand-rock-o:before{content:"\f255"}.fa-hand-stop-o:before,.fa-hand-paper-o:before{content:"\f256"}.fa-hand-scissors-o:before{content:"\f257"}.fa-hand-lizard-o:before{content:"\f258"}.fa-hand-spock-o:before{content:"\f259"}.fa-hand-pointer-o:before{content:"\f25a"}.fa-hand-peace-o:before{content:"\f25b"}.fa-trademark:before{content:"\f25c"}.fa-registered:before{content:"\f25d"}.fa-creative-commons:before{content:"\f25e"}.fa-gg:before{content:"\f260"}.fa-gg-circle:before{content:"\f261"}.fa-tripadvisor:before{content:"\f262"}.fa-odnoklassniki:before{content:"\f263"}.fa-odnoklassniki-square:before{content:"\f264"}.fa-get-pocket:before{content:"\f265"}.fa-wikipedia-w:before{content:"\f266"}.fa-safari:before{content:"\f267"}.fa-chrome:before{content:"\f268"}.fa-firefox:before{content:"\f269"}.fa-opera:before{content:"\f26a"}.fa-internet-explorer:before{content:"\f26b"}.fa-tv:before,.fa-television:before{content:"\f26c"}.fa-contao:before{content:"\f26d"}.fa-500px:before{content:"\f26e"}.fa-amazon:before{content:"\f270"}.fa-calendar-plus-o:before{content:"\f271"}.fa-calendar-minus-o:before{content:"\f272"}.fa-calendar-times-o:before{content:"\f273"}.fa-calendar-check-o:before{content:"\f274"}.fa-industry:before{content:"\f275"}.fa-map-pin:before{content:"\f276"}.fa-map-signs:before{content:"\f277"}.fa-map-o:before{content:"\f278"}.fa-map:before{content:"\f279"}.fa-commenting:before{content:"\f27a"}.fa-commenting-o:before{content:"\f27b"}.fa-houzz:before{content:"\f27c"}.fa-vimeo:before{content:"\f27d"}.fa-black-tie:before{content:"\f27e"}.fa-fonticons:before{content:"\f280"}
diff --git a/doc/book/_FontAwesome/fonts/FontAwesome.ttf b/doc/book/_FontAwesome/fonts/FontAwesome.ttf
new file mode 100644
index 00000000..d7994e13
Binary files /dev/null and b/doc/book/_FontAwesome/fonts/FontAwesome.ttf differ
diff --git a/doc/book/_FontAwesome/fonts/fontawesome-webfont.eot b/doc/book/_FontAwesome/fonts/fontawesome-webfont.eot
new file mode 100644
index 00000000..a30335d7
Binary files /dev/null and b/doc/book/_FontAwesome/fonts/fontawesome-webfont.eot differ
diff --git a/doc/book/_FontAwesome/fonts/fontawesome-webfont.svg b/doc/book/_FontAwesome/fonts/fontawesome-webfont.svg
new file mode 100644
index 00000000..6fd19abc
--- /dev/null
+++ b/doc/book/_FontAwesome/fonts/fontawesome-webfont.svg
@@ -0,0 +1,640 @@
+
+
+
\ No newline at end of file
diff --git a/doc/book/_FontAwesome/fonts/fontawesome-webfont.ttf b/doc/book/_FontAwesome/fonts/fontawesome-webfont.ttf
new file mode 100644
index 00000000..d7994e13
Binary files /dev/null and b/doc/book/_FontAwesome/fonts/fontawesome-webfont.ttf differ
diff --git a/doc/book/_FontAwesome/fonts/fontawesome-webfont.woff b/doc/book/_FontAwesome/fonts/fontawesome-webfont.woff
new file mode 100644
index 00000000..6fd4ede0
Binary files /dev/null and b/doc/book/_FontAwesome/fonts/fontawesome-webfont.woff differ
diff --git a/doc/book/_FontAwesome/fonts/fontawesome-webfont.woff2 b/doc/book/_FontAwesome/fonts/fontawesome-webfont.woff2
new file mode 100644
index 00000000..5560193c
Binary files /dev/null and b/doc/book/_FontAwesome/fonts/fontawesome-webfont.woff2 differ
diff --git a/doc/book/backend.html b/doc/book/backend.html
new file mode 100644
index 00000000..2374ca01
--- /dev/null
+++ b/doc/book/backend.html
@@ -0,0 +1,129 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Via the concept of a backend we can abstract over the platform we will execute
+or optimize a network on. The construction of a backend is trivial. The backend
+is passed to the Solver, (one backend for network and one for the
+objectve). The Solver than executes all operations on the provided backend.
In the previous chapters, we learned that in Leaf everything is build by
+layers and that the constructed thing is again a layer, which means it can
+function as a new building block for something bigger. This is possible, because
+a Layer can implement any behavior as long as it takes an input and produces
+an output.
+
In 2.1 Layer Lifecycle
+we have seen, that only one LayerConfig can be used to turn it via
+Layer::from_config into an actual Layer. But as Deep Learning relies on
+chaining multiple layers together, we need a Layer, who implements this
+behavior for us.
+
Enter the container layers.
+
Networks via the Sequential layer
+
A Sequential Layer is a layer of type container layer. The config of a
+container layer has a special method called,
+.add_layer which takes one LayerConfig and adds it to an ordered list in the
+SequentialConfig.
+
When turning a SequentialConfig into a Layer by passing the config to
+Layer::from_config, the behavior of the Sequential is to initialize all the
+layers which were added via .add_layer and connect the layers with each other.
+This means, the output of one layer becomes the input of the next layer in the
+list.
+
The input of a sequential Layer becomes the input of the
+first layer in the sequential worker, the sequential worker then takes care
+of passing the input through all the layers and the output of the last layer
+then becomes the output of the Layer with the sequential worker. Therefore
+a sequential Layer fulfills the requirements of a Layer - take an input,
+return an output.
As a sequential layer is like any other layer, we can use sequential layers as
+building blocks for larger networks. Important building blocks of a network can
+be grouped into a sequential layer and published as a crate for others to use.
So far, there is only the sequential layer, but other container layers, with
+slightly different behaviors are conceivable. For example a parallel or
+concat layer in addition to the sequential layer.
+
How to 'train' or optimize the constructed network is topic of chapter 3.
+Solvers
A layer in Leaf can implement any behavior as long as it takes an input and
+produces an output. As Leaf is new, there are still many valuable layers that
+are not yet implemented. This is why this chapter shows how you can add new
+layers to Leaf.
+
A not exclusive list of steps to take in order to implement a new layer:
+
+
The Rust compiler is also very helpful with pointing out the necessary steps
+for implementing a new layer struct. It might be beneficial to start the
+implementation of a new layer from a copied file of an already existing layer.
+
+
+
+
Decide to which of the five types
+the new layer belongs. This decides under which directory to put the layer
+implementation in the Leaf project.
+
+
+
Create the Layer worker struct.
+
+
+
Expose the Layer worker struct in the mod.rs of the layer type directory.
+
+
+
Expose the Layer worker struct in the mod.rs of the /layers directory.
+
+
+
Implement ILayer and its trait boundaries for the new Layer worker struct.
+
+
+
Add the new layer to the LayerType in layer.rs and add the matching
+for .support_in_place and .worker_from_config.
+
+
+
If the new layer relies on a collenchyma operation, also add the collenchyma
+trait boundary.
+
+
+
Add documentation and serialization to the new layer.
+
+
+
(optional) Depending on how complex the layer is, you might also add tests and more
+advanced implementations for its .from_config, .reshape or other helper
+methods.
A layer is the highest-level building block in a (Deep) Neural Network. A layer
+is a container that usually receives weighted input, transforms it and returns
+the result as output to the next layer. A layer usually contains one type of
+function like ReLU, pooling, convolution etc. so that it can be easily compared
+to other parts of the network. The first and last layers in a network are called
+input and output layers, respectively, and all layers in between are called
+hidden layers.
+
In Leaf
+
In Leaf, a layer is very similar to the general understanding of a layer. A layer
+in Leaf, like a layer in a (Deep) Neural Network,
+
+
is the highest-level building block
+
needs to receive input, might transform it and needs to return the result
+
should be uniform (it does one type of function)
+
+
Additionally to a Neural Network layer, a Leaf layer can implement any
+functionality, not only those related to Neural Networks like ReLU, pooling,
+LSTM, etc. For example, the Sequential layer in Leaf, allows it to connect
+multiple layers, creating a network.
+
Network
+
In General
+
A network, also often called Neural Network (NN) or Artificial Neural Network
+(ANN) is a subset of Machine Learning methods.
+
A not exhaustive list of other Machine Learning methods:
+Linear Regression, SVM, Genetic/Evolution Algorithms, dynamic programming,
+deterministic algorithmic optimization methods.
+
In Leaf
+
In Leaf, a network means a graph (a connected set) of one or more
+layers. This network can consist of Artificial Neural Network
+methods, other Machine Learning methods or any other (not Machine Learning
+related) methods. As described in 2. Layers a network in Leaf
+is actually a layer which connects other layers.
+
An initialized network is a network, which is ready to be executed, meaning it
+is fully constructed e.g. all necessary memory is allocated on the host or device.
The distributed optimization of networks will (very likely) be managed by a
+standalone crate on top of Leaf. Although distributed optimization will not be a
+core part of Leaf itself, we will cover the topic of distributed optimization
+with Leaf here in this chapter of the book.
Our life is frittered away by detail. Simplify, simplify. -
+Henry David Thoreau
+
+
This short book teaches you how you can build machine learning applications (with
+Leaf).
+
Leaf is a Machine Intelligence Framework engineered by hackers, not scientists.
+It has a very simple API consisting of Layers and Solvers, with which
+you can build classical machine as well as deep learning and other fancy machine
+intelligence applications. Although Leaf is just a few months old,
+thanks to Rust and Collenchyma it is already one of the fastest machine intelligence
+frameworks available.
+
Leaf was inspired by the brilliant people behind TensorFlow, Torch, Caffe,
+Rust and numerous research papers and brings modularity, performance and
+portability to deep learning.
To make the most of the book, a basic understanding of the fundamental concepts
+of machine and deep learning is recommended. Good resources to get you from
+zero to almost-ready-to-build-machine-learning-applications:
Both machine and deep learning are really easy with Leaf.
+
Construct a Network by chaining Layers.
+Then optimize the network by feeding it examples.
+This is why Leaf's entire API consists of only two concepts: Layers
+and Solvers. Use layers to construct almost any kind of model: deep,
+classical, stochastic or hybrids, and solvers for executing and optimizing the
+model.
+
This is already the entire API for machine learning with Leaf. To learn how
+this is possible and how to build machine learning applications, refer to chapters
+2. Layers and 3. Solvers. Enjoy!
+
Benefits+
+
Leaf was built with three concepts in mind: accessibility/simplicity,
+performance and portability. We want developers and companies to be able to
+run their machine learning applications anywhere: on servers, desktops,
+smartphones and embedded devices. Any combination of platform and
+computation language (OpenCL, CUDA, etc.) is a first class citizen in Leaf.
+
We coupled portability with simplicity, meaning you can deploy your machine
+learning applications to almost any machine and device with no code changes.
+Learn more at chapter 4. Backend or at the
+Collenchyma Github repository.
Alongside this book you can also read the Rust API documentation if
+you would like to use Leaf as a crate, write a library on top of it or
+just want a more low-level overview.
In chapter 2. Layers we saw how to
+construct a simple Layer from a LayerConfig. In this chapter, we take
+a closer look at what happens inside Leaf when initializing a Layer and when running its
+.forward and .backward methods. In the next chapter 2.2 Create a Network we
+apply our knowledge to construct deep networks with the container layer.
+
The most important methods of a Layer are initialization (::from_config), .forward and .backward.
+They basically describe the entire API, so let's take a closer look at what happens inside Leaf when these methods are called.
+
Initialization
+
A layer is constructed from a LayerConfig with the Layer::from_config
+method, which returns a fully initialized Layer.
In the example above, the first layer has a Sigmoid worker
+(LayerType::Sigmoid) and the second layer has a Sequential worker.
+Although both ::from_config methods return a Layer, the behavior of
+that Layer depends on the LayerConfig it was constructed with. The
+Layer::from_config internally calls the worker_from_config method, which
+constructs the specific worker defined by the LayerConfig.
The layer-specific ::from_config (if available or needed) then takes care of
+initializing the worker struct, allocating memory for weights and so on.
+
If the worker is a container layer, its ::from_config takes
+care of initializing all the LayerConfigs it contains (which were added via its
+.add_layer method) and connecting them in the order they were provided.
+
Every .forward or .backward call that is made on the returned Layer is
+run by the internal worker.
+
Forward
+
The forward method of a Layer threads the input through the constructed
+network and returns the output of the network's final layer.
+
The .forward method does three things:
+
+
Reshape the input data if necessary
+
Sync the input/weights to the device where the computation happens. This step
+removes the need for the worker layer to care about memory synchronization.
+
Call the forward method of the internal worker layer.
+
+
If the worker layer is a container layer, the .forward method
+takes care of calling the .forward methods of its managed
+layers in the right order.
+
Backward
+
The .backward method of a Layer works similarly to .forward, apart from
+needing to reshape the input. The .backward method computes
+the gradient with respect to the input as well as the gradient w.r.t. the parameters. However,
+the method only returns the input gradient because that is all that is needed to compute the
+gradient of the entire network via the chain rule.
+
If the worker layer is a container layer, the .backward method
+takes care of calling the .backward_input and
+.backward_parameter methods of its managed layers in the right order.