Stream Transaction HTTP docs

Documentation/Books/HTTP/AqlQueryCursor/README.md

@@ -1,9 +1,6 @@
 HTTP Interface for AQL Query Cursors
 ====================================
 
-Database Cursors
-----------------
-
 This is an introduction to ArangoDB's HTTP Interface for Queries. Results of AQL
 and simple queries are returned as cursors in order to batch the communication
 between server and client. Each call returns a number of documents in a batch

Documentation/Books/HTTP/SUMMARY.md

@@ -51,6 +51,8 @@
   * [ArangoSearch Views](Views/ArangoSearch.md)
 * [Analyzers](Analyzers/README.md)
 * [Transactions](Transaction/README.md)
+  * [Stream Transactions](Transaction/StreamTransaction.md)
+  * [JavaScript Transactions](Transaction/JsTransaction.md)
 * [Replication](Replications/README.md)
   * [Replication Dump](Replications/ReplicationDump.md)
   * [Replication Logger](Replications/ReplicationLogger.md)

Documentation/Books/HTTP/Transaction/JsTransaction.md

@@ -0,0 +1,24 @@
+HTTP Interface for JavaScript Transactions
+==========================================
+
+ArangoDB's JS-transactions are executed on the server. Transactions can be 
+initiated by clients by sending the transaction description for execution to
+the server.
+
+JS-Transactions in ArangoDB do not offer separate *BEGIN*, *COMMIT* and *ROLLBACK*
+operations. Instead, ArangoDB JS-transactions are described by a JavaScript function, 
+and the code inside the JavaScript function will then be executed transactionally.
+
+At the end of the function, the transaction is automatically committed, and all
+changes done by the transaction will be persisted. If an exception is thrown
+during transaction execution, all operations performed in the transaction are
+rolled back.
+
+For a more detailed description of how transactions work in ArangoDB please
+refer to [Transactions](../../Manual/Transactions/index.html). 
+
+<!-- RestTransactionHandler.cpp -->
+
+@startDocuBlock post_api_transaction
+
+

Documentation/Books/HTTP/Transaction/README.md

@@ -4,20 +4,33 @@ HTTP Interface for Transactions
 ### Transactions
 
 ArangoDB's transactions are executed on the server. Transactions can be 
-initiated by clients by sending the transaction description for execution to
-the server.
+executed by clients in two different ways:
 
-Transactions in ArangoDB do not offer separate *BEGIN*, *COMMIT* and *ROLLBACK*
-operations as they are available in many other database products. 
-Instead, ArangoDB transactions are described by a JavaScript function, and the 
-code inside the JavaScript function will then be executed transactionally.
-At the end of the function, the transaction is automatically committed, and all
-changes done by the transaction will be persisted. If an exception is thrown
-during transaction execution, all operations performed in the transaction are
-rolled back.
+1. Via the [Stream Transaction](StreamTransaction.md) API
+2. Via the [JavaScript Transaction](JsTransaction.md) API
 
-For a more detailed description of how transactions work in ArangoDB please
+The difference between these two is not difficult to understand, a short primer 
+is listed below. 
+For a more detailed description of how transactions work in ArangoDB and
+what guarantees ArangoDB can deliver please
 refer to [Transactions](../../Manual/Transactions/index.html). 
 
-<!-- js/actions/api-transaction.js -->
-@startDocuBlock post_api_transaction
+
+### Stream Transactions
+
+The [Stream Transactions](StreamTransaction.md) allows you to perform a multi-document transaction 
+with individual begin and commit / abort commands. This is similar to
+the way traditional RDBMS do it with *BEGIN*, *COMMIT* and *ROLLBACK* operations.
+
+This the recommended API for larger transactions. However the client is responsible
+for making sure that the transaction is committed or aborted when it is no longer needed,
+to avoid taking up resources.
+
+###  JavaScript Transactions
+
+The [JS-Transaction API](JsTransaction.md) allows you to send the server
+a dedicated peace of JavaScript code (i.e. a function), which will be executed transactionally.
+
+At the end of the function, the transaction is automatically committed, and all
+changes done by the transaction will be persisted. No interaction is required by 
+the client beyond the initial start request.

Documentation/Books/HTTP/Transaction/StreamTransaction.md

@@ -0,0 +1,64 @@
+HTTP Interface for Stream Transactions
+======================================
+
+*Stream Transactions* allows you to perform a multi-document transaction 
+with individual begin and commit / abort commands. This is similar to
+the way traditional RDBMS do it with *BEGIN*, *COMMIT* and *ROLLBACK* operations.
+
+To use a stream transaction a client first sends the (configuration)[#begin-a-transaction]
+of the transaction to the ArangoDB server.
+
+{% hint 'info' %}
+Contrary to the [JS-Transaction](JsTransaction.md) the definition of this 
+transaction must only contain the collections which are going to be used
+and (optionally) the various transaction options supported by ArangoDB.
+No *action* attribute is supported.
+{% endhint %}
+
+The stream transaction API works in *conjunction* with other APIs in ArangoDB.
+To use the transaction for a supported operation a client needs to specify
+the transaction identifier in the *x-arango-trx-id* header on each request.
+This will automatically cause these operations to use the specified transaction.
+
+Supported transactional API operations include:
+
+1. All operations in the [Document API](../Document/WorkingWithDocuments.md)
+2. Number of documents via the [Collection API](../Collection/Getting.md#return-number-of-documents-in-a-collection)
+3. Truncate a collection via the [Collection API](../Collection/Getting.md#return-number-of-documents-in-a-collection)
+4. Create an AQL cursor via the [Cursor API](../AqlQueryCursor/AccessingCursors.md)
+
+Note that a client *always needs to start the transaction first* and it is required to
+explicitly specify the collections used for write accesses. The client is responsible
+for making sure that the transaction is committed or aborted when it is no longer needed.
+This avoids taking up resources on the ArangoDB server.
+
+For a more detailed description of how transactions work in ArangoDB please
+refer to [Transactions](../../Manual/Transactions/index.html). 
+
+Begin a Transaction
+-------------------
+
+<!-- RestTransactionHandler.cpp -->
+
+@startDocuBlock post_api_transaction_begin
+
+Check Status of a Transaction
+-----------------------------
+
+@startDocuBlock get_api_transaction
+
+Commit or Abort a Transaction
+-----------------------------
+
+Committing or aborting a running transaction must be done by the client.
+It is *bad practice* to not commit or abort a transaction once you are done
+using it. It will force the server to keep resources and collection locks 
+until the entire transaction times out.
+
+<!-- RestTransactionHandler.cpp -->
+
+@startDocuBlock put_api_transaction
+
+<!-- RestTransactionHandler.cpp -->
+
+@startDocuBlock delete_api_transaction

Documentation/Books/Manual/ReleaseNotes/NewFeatures35.md

@@ -247,6 +247,19 @@ The HTTP API for running Foxx service tests now supports a `filter` attribute,
 which can be used to limit which test cases should be executed.
 
 
+### Stream Transaction API
+
+There is a new HTTP API for transactions. This API allows clients to add operations to a
+transaction in a streaming fashion. A transaction can consist of a series of supported
+transactional operations, followed by a commit or abort command.
+This allows clients to construct larger transactions in a more efficent way then
+with the JavaScript based transactions.
+
+Note that this requires client applications to abort transactions which are no 
+longer necessary. Otherwise resources and locks acquired by the transactions
+will hang around until the server decides to garbage collect them.
+
+
 Web interface
 -------------
 

Documentation/Books/Manual/Transactions/Limitations.md

@@ -5,7 +5,7 @@ In General
 ----------
 
 Transactions in ArangoDB have been designed with particular use cases 
-in mind. They will be mainly useful for short and small data retrieval 
+in mind. They will be mainly useful for *short and small* data retrieval 
 and/or modification operations.
 
 The implementation is not optimized for very long-running or very voluminous
@@ -58,6 +58,8 @@ Please refer to [Locking and Isolation](LockingAndIsolation.md) for more details
 In Clusters
 -----------
 
+TODO: fix this section up
+
 Using a single instance of ArangoDB, multi-document / multi-collection queries
 are guaranteed to be fully ACID. This is more than many other NoSQL database
 systems support. In cluster mode, single-document operations are also fully ACID.
@@ -74,7 +76,7 @@ With RocksDB storage engine
 The following restrictions and limitations do not apply to JavaScript
 transactions, since their intended use case is for smaller transactions
 with full transactional guarantees. So the following only applies
-to AQL transactions and transactions created through the document API.
+to AQL queries and transactions created through the document API (i.e. batch operations).
 {% endhint %}
 
 Data of ongoing transactions is stored in RAM. Transactions that get too big 

Documentation/DocuBlocks/Rest/Transactions/delete_api_transaction.md

@@ -0,0 +1,87 @@
+
+@startDocuBlock delete_api_transaction
+@brief abort a server-side transaction
+
+@RESTHEADER{DELETE /_api/transaction/{transaction-id}, Abort transaction}
+
+@RESTURLPARAMETERS
+
+@RESTURLPARAM{transaction-id,string,required}
+The transaction identifier,
+
+@RESTDESCRIPTION
+Abort a running server-side transaction. Aborting is an idempotent operation. 
+It is not an error to abort a transaction more than once.
+
+If the transaction can be aborted, *HTTP 200* will be returned. 
+The returned JSON object has the following properties:
+
+- *error*: boolean flag to indicate if an error occurred (*false*
+  in this case)
+
+- *code*: the HTTP status code
+
+- *result*: result containing
+    - *id*: the identifier of the transaction
+    - *status*: containing the string 'aborted'
+
+If the transaction cannot be found, aborting is not allowed or the 
+transaction was already committed, the server
+will respond with *HTTP 400*, *HTTP 404* or *HTTP 409*.
+
+The body of the response will then contain a JSON object with additional error
+details. The object has the following attributes:
+
+- *error*: boolean flag to indicate that an error occurred (*true* in this case)
+
+- *code*: the HTTP status code
+
+- *errorNum*: the server error number
+
+- *errorMessage*: a descriptive error message
+
+
+@RESTRETURNCODES
+
+@RESTRETURNCODE{200}
+If the transaction was aborted,
+*HTTP 200* will be returned.
+
+@RESTRETURNCODE{400}
+If the transaction cannot be aborted, the server
+will respond with *HTTP 400*.
+
+@RESTRETURNCODE{404}
+If the transaction was not found, the server
+will respond with *HTTP 404*.
+
+@RESTRETURNCODE{409}
+If the transaction was already committed, the server
+will respond with *HTTP 409*.
+
+@EXAMPLES
+
+Aborting a transaction:
+
+@EXAMPLE_ARANGOSH_RUN{RestTransactionBeginCommit}
+    const cn = "products";
+    db._drop(cn);
+    db._create(cn);
+    let body = {
+      collections: {
+        read : cn
+      }
+    };
+    let trx = db._createTransaction(body);
+    let url = "/_api/transaction/" + trx.id();
+
+    var response = logCurlRequest('DELETE', url);
+    assert(response.code === 200);
+
+    logJsonResponse(response);
+
+  ~ db._drop(cn);
+@END_EXAMPLE_ARANGOSH_RUN
+
+@endDocuBlock
+

Documentation/DocuBlocks/Rest/Transactions/get_api_transaction.md

@@ -0,0 +1,59 @@
+
+@startDocuBlock get_api_transaction
+@brief Fetch status of a server-side transaction
+
+@RESTHEADER{GET /_api/transaction/{transaction-id}, Get transaction status}
+
+@RESTURLPARAMETERS
+
+@RESTURLPARAM{transaction-id,string,required}
+The transaction identifier.
+
+@RESTDESCRIPTION
+The result is an object describing the status of the transaction. 
+It has at least the following attributes:
+
+- *id*: the identifier of the transaction
+
+- *status*: the status of the transaction. One of "running", "committed" or "aborted".
+
+@RESTRETURNCODES
+
+@RESTRETURNCODE{200}
+If the transaction is fully executed and committed on the server,
+*HTTP 200* will be returned.
+
+@RESTRETURNCODE{400}
+If the transaction identifier specified is either missing or malformed, the server
+will respond with *HTTP 400*.
+
+@RESTRETURNCODE{404}
+If the transaction was not found with the specified identifier, the server
+will respond with *HTTP 404*.
+
+@EXAMPLES
+
+Get transaction status
+
+@EXAMPLE_ARANGOSH_RUN{RestTransactionGet}
+    db._drop("products");
+    db._create("products");
+    let body = {
+      collections: {
+        read : "products"
+      }
+    };
+    let trx = db._createTransaction(body);
+    let url = "/_api/transaction/" + trx.id();
+
+    let response = logCurlRequest('GET', url);
+    assert(response.code === 200);
+
+    logJsonResponse(response);
+
+  ~ trx.abort();
+  ~ db._drop("products");
+@END_EXAMPLE_ARANGOSH_RUN
+
+@endDocuBlock
+

Documentation/DocuBlocks/Rest/Transactions/post_api_transaction.md

@@ -30,6 +30,9 @@ attribute if the transaction committed successfully.
 an optional boolean flag that, if set, will force the
 transaction to write all data to disk before returning.
 
+@RESTBODYPARAM{allowImplicit,boolean,optional,boolean}
+Allow reading from undeclared collections.
+
 @RESTBODYPARAM{lockTimeout,integer,optional,int64}
 an optional numeric value that can be used to set a
 timeout for waiting on collection locks. If not specified, a default