Permalink
Browse files

2.3.0 release commit.

  • Loading branch information...
1 parent 4c6aae2 commit 3c5eeef7b0ae3d4c7f05fd14fa063e2ede981e74 @okram okram committed Mar 20, 2013
Showing with 18,719 additions and 13 deletions.
  1. +2 −2 CHANGELOG.textile
  2. +1 −1 blueprints-core/pom.xml
  3. +1 −1 blueprints-dex-graph/pom.xml
  4. +1 −1 blueprints-graph-jung/pom.xml
  5. +1 −1 blueprints-graph-sail/pom.xml
  6. +1 −1 blueprints-neo4j-graph/pom.xml
  7. +1 −1 blueprints-orient-graph/pom.xml
  8. +1 −1 blueprints-rexster-graph/pom.xml
  9. +1 −1 blueprints-sail-graph/pom.xml
  10. +1 −1 blueprints-test/pom.xml
  11. +78 −0 doc/Acknowledgments.html
  12. +133 −0 doc/Batch-Implementation.html
  13. +132 −0 doc/Code-Examples.html
  14. +83 −0 doc/Desired-Implementations.html
  15. +145 −0 doc/Dex-Implementation.html
  16. +102 −0 doc/Event-Implementation.html
  17. +171 −0 doc/GML-Reader-and-Writer-Library.html
  18. +95 −0 doc/Graph-Indices.html
  19. +84 −0 doc/Graph-Morphisms.html
  20. +96 −0 doc/Graph-Transactions.html
  21. +224 −0 doc/GraphML-Reader-and-Writer-Library.html
  22. +350 −0 doc/GraphSON-Reader-and-Writer-Library.html
  23. +155 −0 doc/Home.html
  24. +104 −0 doc/Id-Implementation.html
  25. +76 −0 doc/Implementations.html
  26. +69 −0 doc/InfiniteGraph-Implementation.html
  27. +113 −0 doc/JUNG-Ouplementation.html
  28. +110 −0 doc/Neo4j-Implementation.html
  29. +111 −0 doc/Neo4jBatch-Implementation.html
  30. +77 −0 doc/Neo4jHa-Implementation.html
  31. +110 −0 doc/OrientDB-Implementation.html
  32. +72 −0 doc/Ouplementations.html
  33. +93 −0 doc/Partition-Implementation.html
  34. +80 −0 doc/Property-Graph-Implementations.html
  35. +160 −0 doc/Property-Graph-Model-Test-Suite.html
  36. +91 −0 doc/Property-Graph-Model.html
  37. +201 −0 doc/PropertyGraphSail-Ouplementation.html
  38. +78 −0 doc/ReadOnly-Implementation.html
  39. +410 −0 doc/Release-Notes.html
  40. +97 −0 doc/Rexster-Implementation.html
  41. +146 −0 doc/Sail-Implementation.html
  42. +234 −0 doc/Sail-Ouplementation.html
  43. +76 −0 doc/The-Benefits-of-Blueprints.html
  44. +109 −0 doc/The-Major-Differences-Between-Blueprints-1.x-and-2.x.html
  45. +105 −0 doc/TinkerGraph.html
  46. +85 −0 doc/Vertex-Query.html
  47. +2,467 −0 doc/css/bootstrap.css
  48. +141 −0 doc/css/dialog.css
  49. +537 −0 doc/css/editor.css
  50. +669 −0 doc/css/gollum.css
  51. +69 −0 doc/css/ie7.css
  52. +316 −0 doc/css/template.css
  53. +1,084 −0 doc/javascript/editor/gollum.editor.js
  54. +168 −0 doc/javascript/editor/langs/asciidoc.js
  55. +103 −0 doc/javascript/editor/langs/creole.js
  56. +211 −0 doc/javascript/editor/langs/markdown.js
  57. +173 −0 doc/javascript/editor/langs/org.js
  58. +111 −0 doc/javascript/editor/langs/pod.js
  59. +74 −0 doc/javascript/editor/langs/rdoc.js
  60. +175 −0 doc/javascript/editor/langs/textile.js
  61. +206 −0 doc/javascript/gollum.dialog.js
  62. +191 −0 doc/javascript/gollum.js
  63. +54 −0 doc/javascript/gollum.placeholder.js
  64. +123 −0 doc/javascript/jquery.color.js
  65. +7,179 −0 doc/javascript/jquery.js
  66. +2 −2 pom.xml
View
@@ -5,13 +5,13 @@ h2. Blueprints 2.y.z
!https://github.com/tinkerpop/blueprints/raw/master/doc/images/blueprints-2.png!
-h3. Version 2.3.0 (NOT OFFICIALLY RELEASED YET)
+h3. Version 2.3.0 (March 20, 2013)
```xml
<dependency>
<groupId>com.tinkerpop.blueprints</groupId>
<artifactId>blueprints-core</artifactId>
- <version>2.3.0-SNAPSHOT</version>
+ <version>2.3.0</version>
</dependency>
```
View
@@ -6,7 +6,7 @@
<parent>
<groupId>com.tinkerpop.blueprints</groupId>
<artifactId>blueprints</artifactId>
- <version>2.3.0-SNAPSHOT</version>
+ <version>2.3.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<artifactId>blueprints-core</artifactId>
@@ -5,7 +5,7 @@
<parent>
<groupId>com.tinkerpop.blueprints</groupId>
<artifactId>blueprints</artifactId>
- <version>2.3.0-SNAPSHOT</version>
+ <version>2.3.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<artifactId>blueprints-dex-graph</artifactId>
@@ -5,7 +5,7 @@
<parent>
<groupId>com.tinkerpop.blueprints</groupId>
<artifactId>blueprints</artifactId>
- <version>2.3.0-SNAPSHOT</version>
+ <version>2.3.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<artifactId>blueprints-graph-jung</artifactId>
@@ -5,7 +5,7 @@
<parent>
<groupId>com.tinkerpop.blueprints</groupId>
<artifactId>blueprints</artifactId>
- <version>2.3.0-SNAPSHOT</version>
+ <version>2.3.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<artifactId>blueprints-graph-sail</artifactId>
@@ -5,7 +5,7 @@
<parent>
<groupId>com.tinkerpop.blueprints</groupId>
<artifactId>blueprints</artifactId>
- <version>2.3.0-SNAPSHOT</version>
+ <version>2.3.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<artifactId>blueprints-neo4j-graph</artifactId>
@@ -5,7 +5,7 @@
<parent>
<groupId>com.tinkerpop.blueprints</groupId>
<artifactId>blueprints</artifactId>
- <version>2.3.0-SNAPSHOT</version>
+ <version>2.3.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<artifactId>blueprints-orient-graph</artifactId>
@@ -5,7 +5,7 @@
<parent>
<groupId>com.tinkerpop.blueprints</groupId>
<artifactId>blueprints</artifactId>
- <version>2.3.0-SNAPSHOT</version>
+ <version>2.3.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<artifactId>blueprints-rexster-graph</artifactId>
@@ -5,7 +5,7 @@
<parent>
<groupId>com.tinkerpop.blueprints</groupId>
<artifactId>blueprints</artifactId>
- <version>2.3.0-SNAPSHOT</version>
+ <version>2.3.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<artifactId>blueprints-sail-graph</artifactId>
View
@@ -5,7 +5,7 @@
<parent>
<groupId>com.tinkerpop.blueprints</groupId>
<artifactId>blueprints</artifactId>
- <version>2.3.0-SNAPSHOT</version>
+ <version>2.3.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<artifactId>blueprints-test</artifactId>
View
@@ -0,0 +1,78 @@
+<!DOCTYPE html>
+<html>
+<head>
+ <meta http-equiv="Content-type" content="text/html;charset=utf-8">
+ <link rel="stylesheet" type="text/css" href=" css/gollum.css" media="all">
+ <link rel="stylesheet" type="text/css" href=" css/editor.css" media="all">
+ <link rel="stylesheet" type="text/css" href=" css/dialog.css" media="all">
+ <link rel="stylesheet" type="text/css" href=" css/template.css" media="all">
+
+ <!--[if IE 7]>
+ <link rel="stylesheet" type="text/css" href=" css/ie7.css" media="all">
+ <![endif]-->
+
+ <script type="text/javascript" src=" css/jquery.js"></script>
+ <script type="text/javascript" src=" css/gollum.js"></script>
+ <script type="text/javascript" src=" css/gollum.dialog.js"></script>
+ <script type="text/javascript" src=" css/gollum.placeholder.js"></script>
+
+ <script type="text/javascript"
+ src=" css/editor/gollum.editor.js"></script>
+ <title>Acknowledgments</title>
+</head>
+<body>
+
+<div id="wiki-wrapper" class="page">
+<div id="head"><h3><a href="../../index.html">TinkerPop 2.3.0</a> - <a href="Home.html">Blueprints</a></h3>
+ <h1>Acknowledgments</h1>
+ <ul class="actions">
+ <li class="minibutton"><a href="/pages"
+ class="action-all-pages">All Pages</a></li>
+ <li class="minibutton" class="jaws">
+ <a href="#" id="minibutton-new-page">New Page</a></li>
+ <li class="minibutton"><a href="/edit/Acknowledgments"
+ class="action-edit-page">Edit Page</a></li>
+ <li class="minibutton"><a href="/history/Acknowledgments"
+ class="action-page-history">Page History</a></li>
+ </ul>
+ <div id="searchbar">
+ <form action="/search" method="get" id="search-form">
+ <div id="searchbar-fauxtext">
+ <input type="text" name="q" id="search-query" value="Search&hellip;" autocomplete="off">
+ <a href="#" id="search-submit" title="Search this wiki">
+ <span>Search</span>
+ </a>
+ </div>
+ </form>
+ </div></div>
+<div id="wiki-content">
+ <div class="wrap">
+ <div id="wiki-body" class="gollum-textile-content">
+ <div id="template">
+ <p><img width="100" src=" images/blueprints-character-2.png" /></p>
+<p>This section provides a list of the people that have contributed in some way to the creation of Blueprints.</p>
+<ol><li><a href="http://markorodriguez.com">Marko A. Rodriguez</a> &#x2014; designed, developed, tested, and documented Blueprints.</li>
+ <li><a href="http://orientechnologies.com">Luca Garulli</a> &#x2014; developed the OrientDB implementation (<code>OrientGraph</code>).</li>
+ <li><a href="http://fortytwo.net">Joshua Shinavier</a> &#x2014; developed Blueprints Sail (<code>GraphSail</code>).</li>
+ <li><a href="http://github.com/pangloss">Darrick Weibe</a> &#x2014; tests, bug fixes, and transaction work.</li>
+ <li><a href="http://stephen.genoprime.com">Stephen Mallette</a> &#x2014; develops <code>RexsterGraph</code>, the IO packages (i.e. GraphSON) and other components.</li>
+ <li><a href="http://github.com/sgomezvillamor">Sergio G&#xF3;mez Villamor</a> &#x2014; developed the Dex implementation (<code>DexGraph</code>).</li>
+ <li><a href="http://www.linkedin.com/in/pierredewilde">Pierre De Wilde</a> &#x2014; designs and tests new features.</li>
+ <li><a href="http://www.ketrinayim.com/">Ketrina Yim</a> &#x2014; designed the Blueprints logo.</li>
+ <li><a href="http://www.matthiasb.com/">Matthias Broecheler</a> &#x2014; designed many of the TinkerPop 2 <span class="caps">API</span> changes.</li>
+</ol><p>Please review Blueprints&#x2019; <a href="http://github.com/tinkerpop/blueprints/blob/master/pom.xml">pom.xml</a>. Blueprints would not be possible without the work done by others to create these useful packages.</p>
+ </div>
+ </div>
+ </div>
+
+</div>
+<div id="footer">
+ <p id="last-edit">Last edited by <b>joshsh</b>, 2013-03-20 04:57:12</p>
+</div>
+</div>
+
+<script type="text/javascript" src=" css/MathJax/MathJax.js"></script>
+
+
+</body>
+</html>
@@ -0,0 +1,133 @@
+<!DOCTYPE html>
+<html>
+<head>
+ <meta http-equiv="Content-type" content="text/html;charset=utf-8">
+ <link rel="stylesheet" type="text/css" href=" css/gollum.css" media="all">
+ <link rel="stylesheet" type="text/css" href=" css/editor.css" media="all">
+ <link rel="stylesheet" type="text/css" href=" css/dialog.css" media="all">
+ <link rel="stylesheet" type="text/css" href=" css/template.css" media="all">
+
+ <!--[if IE 7]>
+ <link rel="stylesheet" type="text/css" href=" css/ie7.css" media="all">
+ <![endif]-->
+
+ <script type="text/javascript" src=" css/jquery.js"></script>
+ <script type="text/javascript" src=" css/gollum.js"></script>
+ <script type="text/javascript" src=" css/gollum.dialog.js"></script>
+ <script type="text/javascript" src=" css/gollum.placeholder.js"></script>
+
+ <script type="text/javascript"
+ src=" css/editor/gollum.editor.js"></script>
+ <title>Batch Implementation</title>
+</head>
+<body>
+
+<div id="wiki-wrapper" class="page">
+<div id="head"><h3><a href="../../index.html">TinkerPop 2.3.0</a> - <a href="Home.html">Blueprints</a></h3>
+ <h1>Batch Implementation</h1>
+ <ul class="actions">
+ <li class="minibutton"><a href="/pages"
+ class="action-all-pages">All Pages</a></li>
+ <li class="minibutton" class="jaws">
+ <a href="#" id="minibutton-new-page">New Page</a></li>
+ <li class="minibutton"><a href="/edit/Batch-Implementation"
+ class="action-edit-page">Edit Page</a></li>
+ <li class="minibutton"><a href="/history/Batch-Implementation"
+ class="action-page-history">Page History</a></li>
+ </ul>
+ <div id="searchbar">
+ <form action="/search" method="get" id="search-form">
+ <div id="searchbar-fauxtext">
+ <input type="text" name="q" id="search-query" value="Search&hellip;" autocomplete="off">
+ <a href="#" id="search-submit" title="Search this wiki">
+ <span>Search</span>
+ </a>
+ </div>
+ </form>
+ </div></div>
+<div id="wiki-content">
+ <div class="wrap">
+ <div id="wiki-body" class="gollum-textile-content">
+ <div id="template">
+ <p><img src="http://www.stacymakescents.com/wp-content/uploads/haystack-clip-art.gif" width="250px" /></p>
+<pre><code>&lt;dependency&gt;
+ &lt;groupId&gt;com.tinkerpop.blueprints&lt;/groupId&gt;
+ &lt;artifactId&gt;blueprints-core&lt;/artifactId&gt;
+ &lt;version&gt;??&lt;/version&gt;
+&lt;/dependency&gt;</code></pre>
+<p><code>BatchGraph</code> wraps any <code>TransactionalGraph</code> to enable batch loading of a large number of edges and vertices by chunking the entire load into smaller batches and maintaining a memory-efficient vertex cache so that intermediate transactional states can be flushed after each chunk is loaded to release memory.</p>
+<p><code>BatchGraph</code> is <strong><span class="caps">ONLY</span></strong> meant for loading data and does not support any retrieval or removal operations. That is, BatchGraph only supports the following methods:</p>
+<ul><li><code>addVertex()</code> for adding vertices</li>
+ <li><code>addEdge()</code> for adding edges</li>
+ <li><code>getVertex()</code> to be used when adding edges</li>
+ <li>Property getter, setter and removal methods for vertices and edges as well as <code>getId()</code></li>
+</ul><p>An important limitation of <code>BatchGraph</code> is that edge properties can only be set immediately after the edge has been added. If other vertices or edges have been created in the meantime, setting, getting or removing properties will throw exceptions. This is done to avoid caching of edges which would require memory.</p>
+<p><code>BatchGraph</code> wraps <code>TransactionalGraph</code>. To wrap arbitrary graphs, use <code>BatchGraph.wrap()</code> which will additionally wrap non-transactional graphs.</p>
+<p><code>BatchGraph</code> can also automatically set the provided element ids as properties on the respective element. Use <code>setVertexIdKey()</code> and <code>setEdgeIdKey()</code> to set the keys for the vertex and edge properties, respectively. This is useful when the graph implementation ignores supplied ids and allows to make the loaded graph compatible for later wrapping with <code>IdGraph</code> (see <a class="internal present" href="Id-Implementation.html">Id Implementation</a>) when setting the vertex and edge Id keys to <code>IdGraph.ID</code>.</p>
+<p>As an example, suppose we are loading a large number of edges defined by a String array with four entries called <em>quads</em>:</p>
+<ol><li>The out vertex id</li>
+ <li>The in vertex id</li>
+ <li>The label of the edge</li>
+ <li>A string annotation for the edge, i.e. an edge property</li>
+</ol><p>Assuming this array is very large, loading all these edges in a single transaction is likely to exhaust main memory. Furthermore, one would have to rely on the database indexes to retrieve previously created vertices for a given id. <code>BatchGraph</code> addresses both of these issues.</p>
+<pre><code>BatchGraph bgraph = new BatchGraph(graph, VertexIDType.STRING, 1000);
+for (String[] quad : quads) {
+ Vertex[] vertices = new Vertex[2];
+ for (int i=0;i&lt;2;i++) {
+ vertices[i] = bgraph.getVertex(quad[i]);
+ if (vertices[i]==null) vertices[i]=bgraph.addVertex(quad[i]);
+ }
+ Edge edge = bgraph.addEdge(null,vertices[0],vertices[1],quad[2]);
+ edge.setProperty("annotation",quad[3]);
+}</code></pre>
+<p>First, a <code>BatchGraph</code> <em>bgraph</em> is created wrapping an existing <em>graph</em> and setting the id type to <code>VertexIDType.STRING</code> and the batch size to 1000. <code>BatchGraph</code> maintains a mapping from the external vertex ids, in our example the first two entries in the String array describing the edge, to the internal vertex ids assigned by the wrapped graph database. Since this mapping is maintained in memory, it is potentially much faster than the database index. By specifying the <code>VertexIDType</code>, <code>BatchGraph</code> chooses the most memory-efficient mapping data structure and applies compression algorithms if possible. There are four different <code>VertexIDType</code>:</p>
+<ul><li><em><span class="caps">OBJECT</span></em> : For arbitrary object vertex ids. This is the most generic and least space efficient type.</li>
+ <li><em><span class="caps">STRING</span></em> : For string vertex ids. Attempts to apply string compression and prefixing strategies to reduce the memory footprint.</li>
+ <li><em><span class="caps">URL</span></em> : For string vertex ids that parse as URLs. Applies <span class="caps">URL</span> specific compression schemes that are more efficient than generic string compression.</li>
+ <li><em><span class="caps">NUMBER</span></em> : For numeric vertex ids. Uses primitive data structures that requires significantly less memory.</li>
+</ul><p>The last argument in the constructor is the batch size, that is, the number of vertices and edges to load before committing a transaction and starting a new one.</p>
+<p>The <code>for</code> loop then iterates over all the quad String arrays and creates an edge for each by first retrieving or creating the vertex end points and then creating the edge. Note, that we set the edge property immediately after creating the edge. This is required because edges are only kept in memory until the next edge is created for efficiency reasons.</p>
+<h2>Presorting Data</h2>
+<p>In the previous example, there is a big speed advantage if the next edge loaded has the same out vertex as the previous edge. Loading all of the out going edges for a particular vertex at once before moving on to the next out vertex makes optimal use of the cache, whereas loading edges in a random order causes many more writes to and flushes of the cache.</p>
+<p>To take advantage of this, the data can be presorted quickly and efficiently using the linux built-in <a href="http://en.wikipedia.org/wiki/Sort_(Unix)">sort</a> command. Let&#x2019;s say that edges are read from a text file <code>edges.txt</code> with one edge per line:</p>
+<pre><code>4 created 5 weight=1.0
+1 knows 4 weight=1.0
+1 knows 2 weight=0.5
+4 created 3 weight=0.4
+6 created 3 weight=0.2
+1 created 3 weight=0.4</code></pre>
+<p>This file can be sorted before loading with</p>
+<pre><code>vadasg$ sort -S4G -o edges_sorted.txt edges.txt</code></pre>
+<p>The <code>-S4G</code> flag gives sort 4Gb of memory to work with. If the file fits into memory the sort will be very fast; otherwise <code>sort</code> will use scratch space on disk to perform the operation. Although this is not as fast, the linux <code>sort</code> command is highly optimized and is not limited in the size of files it can process. If the input data contain unwanted duplicate lines, using the <code>-u</code> flag will cause <code>sort</code> to remove these duplicate lines during processing.</p>
+<p>The sorted file <code>edges_sorted.txt</code> now has the edges ordered by out vertex:</p>
+<pre><code>1 created 3 weight=0.4
+1 knows 2 weight=0.5
+1 knows 4 weight=1.0
+4 created 3 weight=0.4
+4 created 5 weight=1.0
+6 created 3 weight=0.2</code></pre>
+<p>This way, any given out vertex is kept in the cache for all of its out going edges. The time needed to sort the data is nearly always much less than the loading time saved by maximizing use of the cache, especially for large input data.</p>
+<h2>Incremental Loading</h2>
+<p>The above describes how <code>BatchGraph</code> can be used to load data into a graph under the assumption that the wrapped graph is initially empty. <code>BatchGraph</code> can also be used to incrementally batch load edges and vertices into a graph with existing data. In this case, vertices may already exist for given ids.</p>
+<p>If the wrapped graph does not ignore ids, then enabling incremental batch loading is as simple as calling <code>setLoadingFromScratch(false)</code>, i.e. to disable the assumption that data is loaded into an empty graph. If the wrapped graph does ignore ids, then one has to tell <code>BatchGraph</code> how to find existing vertices for a given id by specifying the vertex id key using <code>setVertexIdKey(uid)</code> where <em>uid</em> is some string for the property key. Also, uid must be <a href="../../blueprints/2.3.0/Graph-Indices.html">key indexed</a> for this to work.</p>
+<pre><code>graph.createKeyIndex("uid",Vertex.class);
+BatchGraph bgraph = new BatchGraph(graph, VertexIDType.STRING, 10000);
+bgraph.setVertexIdKey("uid");
+bgraph.setLoadingFromScratch(false);
+//Load data as shown above</code></pre>
+<p>Note, that incremental batch loading is more expensive than loading from scratch because <code>BatchGraph</code> has to call on the wrapped graph to determine whether a vertex exists for a given id.</p>
+ </div>
+ </div>
+ </div>
+
+</div>
+<div id="footer">
+ <p id="last-edit">Last edited by <b>joshsh</b>, 2013-03-20 04:57:12</p>
+</div>
+</div>
+
+<script type="text/javascript" src=" css/MathJax/MathJax.js"></script>
+
+
+</body>
+</html>
Oops, something went wrong.

0 comments on commit 3c5eeef

Please sign in to comment.