From 819ef5a52e4da7adffd8fcc87900791513fad2f4 Mon Sep 17 00:00:00 2001 From: Amarantha Kulkarni Date: Fri, 7 Sep 2018 13:41:32 -0700 Subject: [PATCH 1/7] Incorporating feedback --- modules/eventing/pages/eventing-adding-function.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/eventing/pages/eventing-adding-function.adoc b/modules/eventing/pages/eventing-adding-function.adoc index b629675662..2fde7d1c8e 100644 --- a/modules/eventing/pages/eventing-adding-function.adoc +++ b/modules/eventing/pages/eventing-adding-function.adoc @@ -61,7 +61,7 @@ Additional controls are now displayed: The controls are: . Click *Deploy*. This displays the *Confirm Deploy Function* dialog. The Feed Boundary determines whether documents previously in existence needs to be included in the Function's activities: the options are *Everything* and *From now*. -The *Everything* option invokes a Function on all the data present in the cluster. +The *Everything* option invokes a Function on all mutations available in the cluster. The *From now* option invokes a Function during future instances of data mutation, post Function deployment. . Click *Deploy* Function. This deploys the Function and returns you to the main Eventing screen. From 0fc9c50902f9a439ecd07808f5d89f413bc6f4e3 Mon Sep 17 00:00:00 2001 From: Amarantha Kulkarni Date: Fri, 7 Sep 2018 13:44:17 -0700 Subject: [PATCH 2/7] Incorporating feedback --- modules/eventing/pages/eventing-api.adoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/modules/eventing/pages/eventing-api.adoc b/modules/eventing/pages/eventing-api.adoc index c0bfa9b31d..a254c47022 100644 --- a/modules/eventing/pages/eventing-api.adoc +++ b/modules/eventing/pages/eventing-api.adoc @@ -3,6 +3,8 @@ [abstract] The Functions REST API, available by default at port 8096, provides the methods available to work with Couchbase Functions. +NOTE: The Functions REST API is a Beta feature intended for development purposes only, do not use them in production; no Enterprise Support is provided for Beta features. + .Functions API [cols="2,3,6"] |=== From e758e54bfe3e5394fa1631ce87034f8931a6f3de Mon Sep 17 00:00:00 2001 From: Amarantha Kulkarni Date: Fri, 7 Sep 2018 13:54:41 -0700 Subject: [PATCH 3/7] incorporating feedback --- .../pages/eventing-language-constructs.adoc | 186 ++++++++---------- 1 file changed, 85 insertions(+), 101 deletions(-) diff --git a/modules/eventing/pages/eventing-language-constructs.adoc b/modules/eventing/pages/eventing-language-constructs.adoc index cd7add3caa..1cd8d1dabc 100644 --- a/modules/eventing/pages/eventing-language-constructs.adoc +++ b/modules/eventing/pages/eventing-language-constructs.adoc @@ -94,8 +94,8 @@ Operations that return values are accessible through a special iterator on which This restricted looping-construct allows support of query-result streaming, and automatic query-cancellation when the iterator goes out of scope. Any variable that is reachable from the scope of the N1QL query can be referred to using *$* syntax in the N1QL statement, where parameters get substituted according to the rules of the named-parameter substitution in the N1QL grammar specification. -IMPORTANT: The N1QL construct is still in development and may have some rough edges and bugs, and may change significantly before the final GA release. -This release version of Couchbase Server 5.5 is intended for development purposes only. +IMPORTANT: The N1QL queries in events are a BETA feature and may have some rough edges and bugs, and may change significantly before the final GA release. +This pre-release version of the feature is intended for development purposes only; no Enterprise Support is provided for Beta features. The iterator is an input iterator (elements are read-only). The keyword _this_ cannot be used in the body of the iterator. @@ -113,6 +113,89 @@ function OnUpdate(doc, meta) { break; // Cancel streaming query by breaking out. } } + +---- +The Function handler code supports N1QL queries. +Top level N1QL keywords, such as SELECT, UPDATE, and INSERT, are available as keywords in Functions. + +During deployment, if a handler code includes an N1QL query, then the system generates a warning message. +[.out]``Warning Message: "Handler uses Beta features. +Do not use in production environments."``However, the warning message does not prevent the Function deployment. + +You must use [.var]`$`, as per N1QL specification, to use a JavaScript variable in the query statement. +The object expressions for substitution are not supported and therefore you cannot use the [.param]`meta.id` expression in the query statement. + +Instead of [.param]`meta.id` expression, you can use `var id = meta.id` in an N1QL query. + +* Invalid N1QL query ++ +---- +DELETE FROM `transactions` WHERE username = $meta.id; +---- + +* Valid N1QL query ++ +---- +var id = meta.id; +DELETE FROM `transactions` WHERE username = $id; +---- + +When you use a N1QL query inside a Function handler, remember to use an escaped identifier for bucket names with special characters (`[.var]`bucket-name``). +Escaped identifiers are surrounded by backticks and support all identifiers in JSON + +For example: + +* If the bucket name is [.param]`beer-sample`, then use the N1QL query such as: ++ +---- +SELECT * FROM `beer-sample` WHERE type... +---- + +* If bucket name is [.param]`beersample`, then use the N1QL query such as: ++ +---- +SELECT * FROM beersample WHERE type ... +---- +The Function handler code supports N1QL queries. +Top level N1QL keywords, such as SELECT, UPDATE, and INSERT, are available as keywords in Functions. + +During deployment, if a handler code includes an N1QL query, then the system generates a warning message. +[.out]``Warning Message: "Handler uses Beta features. +Do not use in production environments."``However, the warning message does not prevent the Function deployment. + +You must use [.var]`$`, as per N1QL specification, to use a JavaScript variable in the query statement. +The object expressions for substitution are not supported and therefore you cannot use the [.param]`meta.id` expression in the query statement. + +Instead of [.param]`meta.id` expression, you can use `var id = meta.id` in an N1QL query. + +* Invalid N1QL query ++ +---- +DELETE FROM `transactions` WHERE username = $meta.id; +---- + +* Valid N1QL query ++ +---- +var id = meta.id; +DELETE FROM `transactions` WHERE username = $id; +---- + +When you use a N1QL query inside a Function handler, remember to use an escaped identifier for bucket names with special characters (`[.var]`bucket-name``). +Escaped identifiers are surrounded by backticks and support all identifiers in JSON + +For example: + +* If the bucket name is [.param]`beer-sample`, then use the N1QL query such as: ++ +---- +SELECT * FROM `beer-sample` WHERE type... +---- + +* If bucket name is [.param]`beersample`, then use the N1QL query such as: ++ +---- +SELECT * FROM beersample WHERE type ... ---- [#handler-signatures] @@ -237,102 +320,3 @@ During the Function deployment step, when the system validates the handler code, Reserved words as a property bindings value image::reserved-words.png[,300] - -== *Support for N1QL in Function Handlers* - -IMPORTANT: The N1QL queries in events are a BETA feature and may have some rough edges and bugs, and may change significantly before the final GA release. -This Beta-release version of Couchbase Server 5.5 is intended for development purposes only; no Enterprise Support is provided for Beta features. - -The Function handler code supports N1QL queries. -Top level N1QL keywords, such as SELECT, UPDATE, and INSERT, are available as keywords in Functions. - -During deployment, if a handler code includes an N1QL query, then the system generates a warning message. -[.out]``Warning Message: "Handler uses Beta features. -Do not use in production environments."``However, the warning message does not prevent the Function deployment. - -You must use [.var]`$`, as per N1QL specification, to use a JavaScript variable in the query statement. -The object expressions for substitution are not supported and therefore you cannot use the [.param]`meta.id` expression in the query statement. - -Instead of [.param]`meta.id` expression, you can use `var id = meta.id` in an N1QL query. - -* Invalid N1QL query -+ ----- -DELETE FROM `transactions` WHERE username = $meta.id; ----- - -* Valid N1QL query -+ ----- -var id = meta.id; -DELETE FROM `transactions` WHERE username = $id; ----- - -When you use a N1QL query inside a Function handler, remember to use an escaped identifier for bucket names with special characters (`[.var]`bucket-name``). -Escaped identifiers are surrounded by backticks and support all identifiers in JSON - -For example: - -* If the bucket name is [.param]`beer-sample`, then use the N1QL query such as: -+ ----- -SELECT * FROM `beer-sample` WHERE type... ----- - -* If bucket name is [.param]`beersample`, then use the N1QL query such as: -+ ----- -SELECT * FROM beersample WHERE type ... ----- - -== Timers - -*Creating a Timer* - -To create a timer use the below syntax: - ----- -createTimer(callback, timestamp, reference, context) ----- - -In the createTimer syntax: - -* *callback* - is the function called when the timer gets triggered. -You need to ensure that the callback function is the top-level function that takes a single argument, the context. -* *timestamp* - is the JavaScript Date object timestamp at which the Function handler code must be executed. -* *reference* - is a unique string that gets passed. -This string helps to identify the timer that is being created. -All callback and references are scoped to the Function definition. -Also, all references must be unique within the Function scope. -When multiple timers are created with the same unique reference, old timers (with the same unique reference) get canceled. -* *context* - is any JavaScript object that can be serialized. -When the timer gets triggered, the context specified during timer creation gets passed to the callback Function. -For optimal performance, the context object payload needs to be lesser than 100 KB. - -A sample *createTimer* language construct is provided for reference. - ----- -createTimer(DocTimerCallback, twoMinsPrior, meta.id, context) ----- - -In the sample construct: - -* *DocTimerCallback* is the name of the function used in the Function handler code. -* *twoMinsPrior* is a JavaScript Date object. -* *meta.id* is a generic reference string that can be used in the Couchbase cluster. -* *context* is the JavaScript object that is used in the Function handler code. - -A few ascpects related to timer construct are listed below: - -* Bindings can be reused in timers. -Bindings, created during the Function definition, can be accessed by the timer constructs in the Function handler code. -* When logging is enabled, timer related logs get captured as part of the Application logs. -* Timers get deleted when the associated Function is deleted or undeployed. -* While using timers, ensure that other applications do not use the metadata bucket. -The metadata bucket stores information about timers and its association with a Function. -Therefore, the metadata bucket should not be deleted or flushed, or the keys be updated. -* With an increase in the usage of timers, the metadata memory assignment must also be increased. -Due to runtime or programmatic errors in the Function handler code, if triggering of a timer fails, then triggering of timer gets permanently blocked. -* For easy debugging, Couchbase recommends enclosing of timers in a try-catch block. -* Timers follow the same timeout semantics as their Parent Functions. -So, if a Function has an execution timeout of 60 seconds, each of the timers created from the Function inherits the same execution timeout value of 60 seconds. From 110990763be166863e0b118c125db54fda32eb34 Mon Sep 17 00:00:00 2001 From: Amarantha Kulkarni Date: Fri, 7 Sep 2018 13:56:10 -0700 Subject: [PATCH 4/7] Incorporate feedback --- modules/eventing/pages/eventing-Terminologies.adoc | 5 ----- 1 file changed, 5 deletions(-) diff --git a/modules/eventing/pages/eventing-Terminologies.adoc b/modules/eventing/pages/eventing-Terminologies.adoc index 6422930089..6f77a8e15b 100644 --- a/modules/eventing/pages/eventing-Terminologies.adoc +++ b/modules/eventing/pages/eventing-Terminologies.adoc @@ -108,8 +108,3 @@ Newly created handlers start in an undeployed state. When a Function gets deleted, the source code implementing the Function, all processing checkpoints, and other artifacts in the metadata provider are purged. Before deleting, make sure you have undeployed the Function. -== Timers - -Timers provide execution of code at a preconfigured clock time or after a specified number of seconds. -Using timers, you can write a simple JavaScript Function handler code to delay or trigger the execution of a Function at specific wall-clock time events. -Timers allow archiving of expired documents at a preconfigured clock time. From 8ce7787488ed5a0f56a5ac8ba7d94a9ed0b59416 Mon Sep 17 00:00:00 2001 From: Amarantha Kulkarni Date: Fri, 7 Sep 2018 13:57:14 -0700 Subject: [PATCH 5/7] Incorporating feedback --- modules/eventing/pages/eventing-examples.adoc | 120 ------------------ 1 file changed, 120 deletions(-) diff --git a/modules/eventing/pages/eventing-examples.adoc b/modules/eventing/pages/eventing-examples.adoc index a2df8c2fae..d6c63b5dce 100644 --- a/modules/eventing/pages/eventing-examples.adoc +++ b/modules/eventing/pages/eventing-examples.adoc @@ -353,123 +353,3 @@ image::query-results-ondelete.png[,400] . In the *Query Results* pane notice that user_ids, *user_id4* and *user_id 10* are removed as part of the cascade user delete operation. -== Example 4 - -*Goal*: When a document in an existing bucket is about to expire, a new document is created in a newly created bucket. - -*Implementation*: Write an OnUpdate handler, which runs whenever a document is created or mutated. -The handler calls a timer routine, which executes a callback function, two minutes prior to any document’s established expiration. -This function retrieves a specified value from the document, and stores it in a document of the same name, in a specified target bucket. -The original document in the source bucket is not changed.. - -For this example, the buckets created such as source, target, and metadata buckets, are used. -A new document is created within the source bucket, and this document has its expiration — or Time To Live (TTL) — set to occur ten minutes after the document's creation. - -Python script for this Example is provided for reference. -Using the Couchbase SDK, you can create or modify the document expiration. -In this example, the Couchbase SDK Python client creates a document and sets the document's expiration. - ----- -from couchbase.cluster import Cluster -from couchbase.cluster import PasswordAuthenticator -import time -cluster = Cluster('couchbase://localhost:8091') -authenticator = PasswordAuthenticator('Administrator', 'password') -cluster.authenticate(authenticator) - -cb = cluster.open_bucket('source') -cb.upsert('SampleDocument2', {'a_key': 'a_value'}) -cb.touch('SampleDocument2', ttl=10*60) ----- - -The script imports a Couchbase cluster object, and authenticates against it, using (for demonstration purposes) the Full Administrator username and password (the cluster is assumed to be accessible on localhost). -The script then opens the existing source bucket, and inserts a new document, named *SampleDocument2*, whose body is *{'a_key': 'a_value'}*. - -For information on installing the Couchbase Python SDK, refer to xref:java-sdk::start-using-sdk.adoc[Start Using the SDK]. -For information on using the Couchbase Python SDK to establish bucket-expiration, refer to xref:dotnet-sdk::document-operations.adoc[Document Operations]. - -*Procedure* - -Proceed as follows: - -. Install the Couchbase SDK Python client and from the appropriate folder, start Python. -+ ----- -./python ----- - -. On the Python prompt, enter the provided code. -+ ----- ->>> from couchbase.cluster import Cluster ->>> from couchbase.cluster import PasswordAuthenticator ->>> import time ->>> cluster = Cluster('couchbase://localhost:8091') ->>> authenticator = PasswordAuthenticator('Administrator', 'password') ->>> cluster.authenticate(authenticator) ->>> cb = cluster.open_bucket('source') ->>> cb.upsert('SampleDocument2', {'a_key': 'a_value'}) -OperationResult ->>> cb.touch('SampleDocument2', ttl=10*60) -OperationResult ->>> ----- - -. To verify bucket creation, access the *Buckets* screen from the *Couchbase Web Console* and click the *Document* tab of the *Source* bucket. -The new document gets displayed. -. [Optional Step] Click on a document's id to view the metadata information. -. From the *Couchbase Web Console* > *Eventing* page, click *ADD FUNCTION*, to add a new Function. -The *ADD FUNCTION* dialog appears. -. In the *ADD FUNCTION* dialog, for individual Function elements provide the below information: - ** For the *Source Bucket* drop-down, select *source*. - ** For the *Metadata Bucket* drop-down, select *metadata*. - ** Enter *add_timer_before_expiry* as the name of the Function you are creating in the *FunctionName* text-box. - ** Enter text *Function that adds timer before document expiry*, in the *Description* text-box. - ** For the *Settings* option, use the default values. - ** For the *Bindings* option, add two bindings. -For the first binding specify *source* as the name of the bucket, and specify *src* as the associated value. -For the second binding, specify *target* as the name of the bucket, and specify *tgt* as the associated value. -. After providing all the required information in the *ADD FUNCTION* dialog, click *Next: Add Code*. -The *add_timer_before_expiry* dialog appears. -. The *add_timer_before_expiry* dialog initially contains a placeholder code block. -You will substitute your actual *add_timer_before_expiry code* in this block. -+ -image::casacade_del_withcode.png[,600] - -. Copy the following Function, and paste it in the placeholder code block of *add_timer_before_expiry* dialog. -+ ----- -function OnUpdate(doc, meta) { - if (meta.expiration > 0 ) //do only for those documents that have a non-zero TTL - { - var expiry = new Date(meta.expiration); - // Compute 2 minutes from the TTL timestamp - var twoMinsPrior = new Date(expiry.setMinutes(expiry.getMinutes()-2)); - var context = {docID : meta.id}; - createTimer(DocTimerCallback, twoMinsPrior , meta.id, context); - log('Added Doc Timer to DocId:', meta.id); - } -} -function DocTimerCallback(context) - { - log('DocTimerCallback Executed for DocId:', String(context.docID)); - tgt[context.docID] = "To Be Expired Key's Value is:" + JSON.stringify(src[context.docID]); - log('Doc Timer Executed for DocId', String(context.docID)); - } ----- -+ -After pasting, the screen appears as displayed below: -+ -image::casacade_del_withcode.png[,600] - -. Click *Save*. -. To return to the Eventing screen, click *Eventing* tab. -. From the *Eventing* screen, click *Deploy*. -. In the *Confirm Deploy Function* dialog, select *Everything from the Feed boundary* option. -. Click *Deploy*. -The function is deployed and starts running within a few seconds. -+ -image::cascade_delete_buckets.png[,600] -+ -As a result, a new document — like the original, named *SourceDocument2* — is created, with a value based on that of the original. -After two minutes has elapsed, check the documents within the source bucket: the original *SourceDocument2* is no longer visible, having been removed at its defined expiration-time. From 418b0146b3c9131a041e37df86d28db90d127773 Mon Sep 17 00:00:00 2001 From: Amarantha Kulkarni Date: Fri, 7 Sep 2018 13:59:33 -0700 Subject: [PATCH 6/7] Incorporating feedback --- modules/eventing/pages/eventing-faq.adoc | 20 -------------------- 1 file changed, 20 deletions(-) diff --git a/modules/eventing/pages/eventing-faq.adoc b/modules/eventing/pages/eventing-faq.adoc index fcd5f5e29a..1ad472a5d9 100644 --- a/modules/eventing/pages/eventing-faq.adoc +++ b/modules/eventing/pages/eventing-faq.adoc @@ -142,23 +142,3 @@ All changes from a document are always processed in order. + We block one of the mutations alone and hand it over to the debugger session. The rest of the mutations continue to be serviced by the event handler. - -* Are timers scalable? -+ -Timers get automatically sharded across Eventing nodes and therefore are elastically scalable. -Due to sharding, triggering of timers at or after a specified time interval is guaranteed. -However, triggering of timers may either be on the same node where the time was created, or on a different node. -Relative ordering between two specific timers cannot be maintained. - -* Can I use Debugger to debug timers? -+ -Timers cannot be debugged using the Visual Debugger. - -* What happens when the Function handler code contains a timestamp in the past? -+ -When a Function handler code contains a timestamp in the past, upon a successful Function deployment, the system executes the code in the next available time slot. - -* What is the Timer behavior post reboot? -+ -During a boot operation, all clocks in the cluster nodes get synchronized. -Post-startup, cluster nodes get periodically synchronized using clock synchronization tools such as Network Time Protocol (NTP). From 0352d17816eff8e66cdd8eefc7eb657a7829792b Mon Sep 17 00:00:00 2001 From: Amarantha Kulkarni Date: Fri, 7 Sep 2018 14:07:11 -0700 Subject: [PATCH 7/7] Incorporate feedback --- .../pages/troubleshooting-best-practices.adoc | 21 ------------------- 1 file changed, 21 deletions(-) diff --git a/modules/eventing/pages/troubleshooting-best-practices.adoc b/modules/eventing/pages/troubleshooting-best-practices.adoc index 89902995c5..cdd9fa9c31 100644 --- a/modules/eventing/pages/troubleshooting-best-practices.adoc +++ b/modules/eventing/pages/troubleshooting-best-practices.adoc @@ -76,24 +76,3 @@ Ensure that you configure the script timeout value after carefully evaluating th As a best practice use a combination of try-catch block and the application log options. This way you can monitor, debug and troubleshoot errors during the Function execution. -== What are a few best practices while passing timer related timestamps? - -Perform a timestamp check to avoid triggering of timers during the stale-time period. - -To handle delays during Function backlogs, in the Function handler code, you can program some additional time to allow triggering of timers. -If this additional time period is also breached, then the time status is considered as stale-time. -If your business logic is time-sensitive, then the Function handler code should refrain from triggering of timers during this stale-time period. - -The following code snippet ensures a valid timestamp check is performed before the Function handler code gets executed. - ----- -func callback(context) -{ - //context.my_deadline is the parameter in the timer payload - if new Date().getTime() > context.my_deadline - { - // timestamp is back-dated, do not execute the rest of the timer - return; - } -} -----