Merge remote-tracking branch 'upstream/v0.4' into mkdocs-master

.github/PULL_REQUEST_TEMPLATE.md

@@ -19,7 +19,7 @@ to ensure the following steps have been taken:
 
 ### For documentation related changes:
 - [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
-- [ ] If this PR is a documentation-only change, have you added a `[skip ci]`
+- [ ] If this PR is a documentation-only change, have you added a `[doc only]`
   tag to the first line of your commit message to avoid spending CPU cycles in
   Travis CI when no code, tests, or build configuration are modified?
 

.gitignore

@@ -59,3 +59,5 @@ coverity_tool.tar.gz
 cov-analysis-linux64/
 cov-int.tar.gz
 cov-int/
+
+/site/

.remarkrc.yml

@@ -0,0 +1,17 @@
+
+# Copyright 2019 JanusGraph Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+plugins:
+  lint-list-item-indent: mixed

BUILDING.md

@@ -1,5 +1,4 @@
-Building JanusGraph
---------------
+# Building JanusGraph
 
 Required:
 
@@ -101,3 +100,24 @@ To find the Java binary in your environment, run the appropriate command for you
 * Linux/macOS: `which java`
 * Windows: `for %i in (java.exe) do @echo. %~$PATH:i`
 
+## Building documentation
+
+
+### Updating documentation
+You have check in the actual version of configuration reference. Therefore, you have to run following command:
+
+```bash
+mvn --quiet clean install -DskipTests=true -pl janusgraph-doc -am
+```
+
+### Required dependencies to build the documentation
+MkDocs need to be installed to build and serve the documentation locally.
+
+1. Install `python3` and `pip3` (newest version of pip) 
+    * You can also checkout the installation guide of [material-mkdocs](https://squidfunk.github.io/mkdocs-material/getting-started/)
+2. Install requirements using `pip3 install -r requirements.txt`
+
+### Build and serve documentation
+
+1. To create a test build locally use command `mkdocs build`
+2. To serve the documentation locally use command `mkdocs serve`

CONTRIBUTING.md

@@ -75,11 +75,11 @@ $ git config user.email "my-email@example.com"
 
 ## Create a new branch
 
-First, you need to decide which release branch (e.g., `master`, `0.2`) to
+First, you need to decide which release branch (e.g., `master`, `v0.2`) to
 create the feature branch from. If you intend to add a new feature, then
 `master` is the right branch. Bug fixes however should also be applied to
 other releases, so you should create your feature branch from the release
-branch with the lowest version number that is still active (e.g., `0.2`).
+branch with the lowest version number that is still active (e.g., `v0.2`).
 When in doubt, ask on [janusgraph-dev](https://groups.google.com/forum/#!forum/janusgraph-dev).
 Changes to all release branches will also be merged into `master`.
 
@@ -96,7 +96,7 @@ $ git checkout -b my-new-feature
 ```
 
 > NOTE: This listing assumes that you create the feature branch from `master`.
-> Replace `master` by the name of the release branch (e.g., `0.2`) if you want
+> Replace `master` by the name of the release branch (e.g., `v0.2`) if you want
 > to create the branch from that release branch instead.
 
 ## Develop and test your changes
@@ -143,7 +143,7 @@ you've signed the contribution.
 > `git ci -v` will get you a diff of your commit while writing your commit
 > message.
 
-> Note: If this is a non-code change, e.g. documentation, add `[skip ci]` to the
+> Note: If this is a non-code change, e.g. documentation, add `[doc only]` to the
 > PR subject line. This is to save CPU time on Travis CI, which lets us get more
 > build time for the other changes which actually change the code.
 >
@@ -152,7 +152,7 @@ you've signed the contribution.
 > * when the PR is submitted for review
 > * when the PR is merged to the base branch
 >
-> Having [skip ci] in the commit skips the first one, but the merge commit also
+> Having [doc only] in the commit skips the first one, but the merge commit also
 > needs it, so having it in the title (first line of commit) helps it easily
 > propagate to both places.
 

RELEASING.md

@@ -1,8 +1,6 @@
-Releasing JanusGraph
-====================
+# Releasing JanusGraph
 
-Prerequisites
--------------
+## Prerequisites
 
 The release process has only been tested on Linux and macOS.
 The following tools must be installed.
@@ -199,18 +197,15 @@ Open up pull requests for the version updates.
 It is recommended to add `[full build]` to the commit message so the full suite of compatibility tests will run.
 After the updates are approved and merged, continue on with the release process.
 
-### Preparing Documentation
-
-Prapare documentation from the release commit.
-Use the [`docs/build-and-copy-docs.sh`](docs/build-and-copy-docs.sh) script to build a set of docs for this release and copy them to the cloned `docs.janusgraph.org` repo which you will update later.
-**NOTE:** the `docs/build-and-copy-docs.sh` script by default expects the janusgraph and docs.janusgraph.org repositories to be located at `~/github/janusgraph/janusgraph` and `~/github/janusgraph/docs.janusgraph.org` respectively.
-It is important that the the `latest` symbolic link at the root of docs.janusgraph.org is pointing to the correct version as well.
-To update symbolic link on linux open `docs.janusgraph.org` repository and perform the next command:
-```Shell
-ln -sfn <the highest version> latest
-```
+### Release Documentation update
 
-Once the documentation for the new version has been generated, open a pull request in [docs.janusgraph.org](https://github.com/JanusGraph/docs.janusgraph.org).
+1. Update version-sensitive files in the root and documentation sources in the `docs` subdirectory: 
+    * `docs/changelog.md`
+    * `mkdocs.yml`
+2. Update the configuration reference: `mvn install -DskipTests=true -pl janusgraph-doc -am`
+3. For building documentation: see `building.md`
+4. Zip documentation: `$ zip janusgraph-${JANUSGRAPH_VERSION}-hadoop2-doc.zip site`
+5. You may also need to update the following file in the main repo for any new or updated dependencies: `NOTICE.txt`
 
 Build Release Artifacts
 -----------------------

docs.Dockerfile

@@ -0,0 +1,25 @@
+#
+# Copyright 2019 JanusGraph Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM alpine:3.9
+
+ENV PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/.local/bin
+
+COPY requirements.txt /mkdocs/
+WORKDIR /mkdocs
+VOLUME /mkdocs
+
+RUN apk --no-cache --no-progress add py3-pip \
+  && pip3 install --user -r requirements.txt

docs/advanced-topics/advschema.md

@@ -0,0 +1,154 @@
+Advanced Schema
+===============
+
+This page describes some of the advanced schema definition options that
+JanusGraph provides. For general information on JanusGraph’s schema and
+how to define it, refer to [Schema and Data Modeling](../basics/schema.md).
+
+Static Vertices
+---------------
+
+Vertex labels can be defined as **static** which means that vertices
+with that label cannot be modified outside the transaction in which they
+were created.
+```groovy
+mgmt = graph.openManagement()
+tweet = mgmt.makeVertexLabel('tweet').setStatic().make()
+mgmt.commit()
+```
+
+Static vertex labels are a method of controlling the data lifecycle and
+useful when loading data into the graph that should not be modified
+after its creation.
+
+Edge and Vertex TTL
+-------------------
+
+Edge and vertex labels can be configured with a **time-to-live (TTL)**.
+Edges and vertices with such labels will automatically be removed from
+the graph when the configured TTL has passed after their initial
+creation. TTL configuration is useful when loading a large amount of
+data into the graph that is only of temporary use. Defining a TTL
+removes the need for manual clean up and handles the removal very
+efficiently. For example, it would make sense to TTL event edges such as
+user-page visits when those are summarized after a certain period of
+time or simply no longer needed for analytics or operational query
+processing.
+
+The following storage backends support edge and vertex TTL.
+
+-   Cassandra
+
+-   HBase
+
+### Edge TTL
+
+Edge TTL is defined on a per-edge label basis, meaning that all edges of
+that label have the same time-to-live. Note that the backend must
+support cell level TTL. Currently only Cassandra and HBase support this.
+```groovy
+mgmt = graph.openManagement()
+visits = mgmt.makeEdgeLabel('visits').make()
+mgmt.setTTL(visits, Duration.ofDays(7))
+mgmt.commit()
+```
+
+Note, that modifying an edge resets the TTL for that edge. Also note,
+that the TTL of an edge label can be modified but it might take some
+time for this change to propagate to all running JanusGraph instances
+which means that two different TTLs can be temporarily in use for the
+same label.
+
+### Property TTL
+
+Property TTL is very similar to edge TTL and defined on a per-property
+key basis, meaning that all properties of that key have the same
+time-to-live. Note that the backend must support cell level TTL.
+Currently only Cassandra and HBase support this.
+```groovy
+mgmt = graph.openManagement()
+sensor = mgmt.makePropertyKey('sensor').cardinality(Cardinality.LIST).dataType(Double.class).make()
+mgmt.setTTL(sensor, Duration.ofDays(21))
+mgmt.commit()
+```
+
+As with edge TTL, modifying an existing property resets the TTL for that
+property and modifying the TTL for a property key might not immediately
+take effect.
+
+### Vertex TTL
+
+Vertex TTL is defined on a per-vertex label basis, meaning that all
+vertices of that label have the same time-to-live. The configured TTL
+applies to the vertex, its properties, and all incident edges to ensure
+that the entire vertex is removed from the graph. For this reason, a
+vertex label must be defined as *static* before a TTL can be set to rule
+out any modifications that would invalidate the vertex TTL. Vertex TTL
+only applies to static vertex labels. Note that the backend must support
+store level TTL. Currently only Cassandra and HBase support this.
+```groovy
+mgmt = graph.openManagement()
+tweet = mgmt.makeVertexLabel('tweet').setStatic().make()
+mgmt.setTTL(tweet, Duration.ofHours(36))
+mgmt.commit()
+```
+
+Note, that the TTL of a vertex label can be modified but it might take
+some time for this change to propagate to all running JanusGraph
+instances which means that two different TTLs can be temporarily in use
+for the same label.
+
+Multi-Properties
+----------------
+
+As discussed in [Schema and Data Modeling](../basics/schema.md), JanusGraph supports property keys with
+SET and LIST cardinality. Hence, JanusGraph supports multiple properties
+with the same key on a single vertex. Furthermore, JanusGraph treats
+properties similarly to edges in that single-valued property annotations
+are allowed on properties as shown in the following example.
+```groovy
+mgmt = graph.openManagement()
+mgmt.makePropertyKey('name').dataType(String.class).cardinality(Cardinality.LIST).make()
+mgmt.commit()
+v = graph.addVertex()
+p1 = v.property('name', 'Dan LaRocque')
+p1.property('source', 'web')
+p2 = v.property('name', 'dalaro')
+p2.property('source', 'github')
+graph.tx().commit()
+v.properties('name')
+==> Iterable over all name properties
+```
+These features are useful in a number of applications such as those
+where attaching provenance information (e.g. who added a property, when
+and from where?) to properties is necessary. Support for higher
+cardinality properties and property annotations on properties is also
+useful in high-concurrency, scale-out design patterns as described in
+[Eventually-Consistent Storage Backends](eventual-consistency.md).
+
+Vertex-centric indexes and global graph indexes are supported for
+properties in the same manner as they are supported for edges. Refer to
+[Indexing for Better Performance](../basics/index-performance.md) for information on defining these indexes for edges and
+use the corresponding API methods to define the same indexes for
+properties.
+
+Unidirected Edges
+-----------------
+
+Unidirected edges are edges that can only be traversed in the out-going
+direction. Unidirected edges have a lower storage footprint but are
+limited in the types of traversals they support. Unidirected edges are
+conceptually similar to hyperlinks in the world-wide-web in the sense
+that the out-vertex can traverse through the edge, but the in-vertex is
+unaware of its existence.
+```groovy
+mgmt = graph.openManagement()
+mgmt.makeEdgeLabel('author').unidirected().make()
+mgmt.commit()
+```
+
+Note, that unidirected edges do not get automatically deleted when their
+in-vertices are deleted. The user must ensure that such inconsistencies
+do not arise or resolve them at query time by explicitly checking vertex
+existence in a transaction. See the discussion in [Ghost Vertices](../basics/common-questions.md#ghost-vertices)
+for more information.