From 37c4196555802d65d7e2fefc59f200642b337096 Mon Sep 17 00:00:00 2001 From: Rakhi Prathap Date: Wed, 2 Jul 2025 18:55:29 +0530 Subject: [PATCH 1/6] Add a new object function --- .../n1ql-language-reference/objectfun.adoc | 104 ++++++++++++++++++ 1 file changed, 104 insertions(+) diff --git a/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc b/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc index df694f6d8..2a89c3e43 100644 --- a/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc @@ -281,6 +281,110 @@ LIMIT 1; ---- ==== + +[[fn-obj-filter,OBJECT_FILTER()]] +== OBJECT_FILTER(`expression` [, `options`]) + +=== Description +This function extracts and returns nested fields from an input object that match a specified pattern, while retaining their original hierarchical path structure in the output. +The function is particularly useful when working with complex objects, as it allows you to filter fields based patterns using either regular expressions or exact matches. + +=== Arguments +expression:: An expression representing an object. + +options:: [Optional] A JSON object specifying options for the function. + +=== Options + +[options="header", cols="1a,3a,1a"] +|=== +| Name | Description | Schema + +| **pattern** + +| The pattern to match. +This can be a regular expression or a simple string, depending on the `regex` parameter. +| String + +| **regex** + +| If `TRUE`, the pattern is treated as a regular expression. + +If `FALSE`, the pattern is treated as a simple string. + +*Default:* `TRUE` +| Boolean + +| **arraysubscript** + +| Specifies whether array subscripts are included in field names before applying the filter. + +If `TRUE`, array subscripts are included. +If `FALSE`, array subscripts are replaced by `*`. + +*Default:* `TRUE` + +| Boolean + +| **composites** + +| Specifies whether the pattern should match field names that contain values requiring further processing, such as nested objects or arrays. + +If `TRUE`, the pattern is matched against every level of nested fields. +If `FALSE`, the pattern is matched only against the deepest level of nested fields. + +*Default:* `TRUE` +| Boolean + +| **patternspace** + +| A string literal with two possible values. + +"field":: The pattern is matched against individual field names. + +"path":: The pattern is matched against composite path names. + +*Default:* `"path"` +| String +|=== + +=== Return Value +An object containing only the fields that match the specified pattern. +Non-matching fields are excluded from the output. + +=== Examples + +[[obj-filter-ex1,OBJECT_FILTER() Example 1]] +.Example 1 +==== +.Query +[source,sqlpp] +---- +SELECT OBJECT_FILTER(t, {"pattern":"Business service"}) +FROM `travel-sample`.`inventory`.`hotel` t +WHERE type = 'hotel' +LIMIT 2; +---- +.Results +[source,json] +---- +[ + { + "$1": { + "reviews": [ + { + "ratings": { + "Business service (e.g., internet access)": 4 + } + } + ] + } + }, + { + "$1": null + } +] +---- +==== + +[[obj-filter-ex2,OBJECT_FILTER() Example 2]] + + [[fn-obj-inner-pairs,OBJECT_INNER_PAIRS()]] == OBJECT_INNER_PAIRS(`expression`) From c7af828bdba36f88706b1d09f4b4edd68af7dd0d Mon Sep 17 00:00:00 2001 From: Rakhi Prathap Date: Thu, 3 Jul 2025 10:10:24 +0530 Subject: [PATCH 2/6] examples + formatting fixes --- .../n1ql-language-reference/objectfun.adoc | 39 +++++++++++++++++-- 1 file changed, 35 insertions(+), 4 deletions(-) diff --git a/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc b/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc index 2a89c3e43..d2e032a9e 100644 --- a/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc @@ -287,7 +287,7 @@ LIMIT 1; === Description This function extracts and returns nested fields from an input object that match a specified pattern, while retaining their original hierarchical path structure in the output. -The function is particularly useful when working with complex objects, as it allows you to filter fields based patterns using either regular expressions or exact matches. +This is particularly useful when working with complex objects, as it allows you to filter fields based patterns using either regular expressions or exact matches. === Arguments expression:: An expression representing an object. @@ -317,6 +317,7 @@ If `FALSE`, the pattern is treated as a simple string. | Specifies whether array subscripts are included in field names before applying the filter. If `TRUE`, array subscripts are included. + If `FALSE`, array subscripts are replaced by `*`. *Default:* `TRUE` @@ -327,6 +328,7 @@ If `FALSE`, array subscripts are replaced by `*`. | Specifies whether the pattern should match field names that contain values requiring further processing, such as nested objects or arrays. If `TRUE`, the pattern is matched against every level of nested fields. + If `FALSE`, the pattern is matched only against the deepest level of nested fields. *Default:* `TRUE` @@ -335,9 +337,9 @@ If `FALSE`, the pattern is matched only against the deepest level of nested fiel | **patternspace** + | A string literal with two possible values. -"field":: The pattern is matched against individual field names. +`"field"`: The pattern is matched against individual field names. -"path":: The pattern is matched against composite path names. +`"path"`: The pattern is matched against composite path names. *Default:* `"path"` | String @@ -350,7 +352,7 @@ Non-matching fields are excluded from the output. === Examples [[obj-filter-ex1,OBJECT_FILTER() Example 1]] -.Example 1 +.Filtering by field name ==== .Query [source,sqlpp] @@ -383,6 +385,35 @@ LIMIT 2; ==== [[obj-filter-ex2,OBJECT_FILTER() Example 2]] +.Filtering by full path +You can use <> to generate the full path to a field, and then use that path in the `pattern` parameter. +==== +.Query +[source,sqlpp] +---- +SELECT OBJECT_FILTER(t, { "pattern": "reviews[1].ratings.Service", "regex": false }) +FROM `travel-sample`.`inventory`.`hotel` t +WHERE type = 'hotel' +LIMIT 1; +---- +.Results +[source,json] +---- +[ + { + "$1": { + "reviews": [ + { + "ratings": { + "Service": 3 + } + } + ] + } + } +] +---- +==== [[fn-obj-inner-pairs,OBJECT_INNER_PAIRS()]] From 770b86956136f8700ce4335995397c77f7cedf24 Mon Sep 17 00:00:00 2001 From: Rakhi Prathap Date: Thu, 3 Jul 2025 10:19:07 +0530 Subject: [PATCH 3/6] Minor fix --- modules/n1ql/pages/n1ql-language-reference/objectfun.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc b/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc index d2e032a9e..abfc2e224 100644 --- a/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc @@ -287,7 +287,7 @@ LIMIT 1; === Description This function extracts and returns nested fields from an input object that match a specified pattern, while retaining their original hierarchical path structure in the output. -This is particularly useful when working with complex objects, as it allows you to filter fields based patterns using either regular expressions or exact matches. +This is particularly useful when working with complex objects, as it allows you to filter fields based on patterns using either regular expressions or exact matches. === Arguments expression:: An expression representing an object. From d781c858caa908d6003c46a453b9afae2ce90ffc Mon Sep 17 00:00:00 2001 From: Rakhi Prathap Date: Thu, 10 Jul 2025 12:36:45 +0530 Subject: [PATCH 4/6] Updates object_pairs_nested() --- .../n1ql-language-reference/objectfun.adoc | 85 +++++++++++++++++-- 1 file changed, 78 insertions(+), 7 deletions(-) diff --git a/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc b/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc index abfc2e224..2b749130d 100644 --- a/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc @@ -343,6 +343,14 @@ If `FALSE`, the pattern is matched only against the deepest level of nested fiel *Default:* `"path"` | String + +| **exact** + +| Specifies whether the provided pattern must be an exact match for the field or path (determined by the `patternspace` parameter). + +It is a short-cut for the regular expression start (^) and end ($) anchors, and you can use it even when `regex` is set to `false`. + +*Default:* `FALSE` +| Boolean |=== === Return Value @@ -960,23 +968,67 @@ If FALSE, only the deepest possible nested fields are returned. *Default:* `FALSE` | Boolean +| **types** + +__optional__ +| Determines whether to return field types or values. + +If TRUE, the function returns the name and type of each field. + +If FALSE, the function returns the name and value of each field. + +*Default:* `FALSE` +| Boolean + | **pattern** + __optional__ -| A regular expression used to filter the returned paths. -The pattern is matched against the composite path names, not the individual field names. +| The pattern used to filter the returned paths. + +It can be a regular expression or a simple string, depending on the `regex` parameter. | String -| **types** + +| **regex** + __optional__ -| Determines whether to return field types or values. +| If `TRUE`, the pattern is treated as a regular expression. -If TRUE, the function returns the name and type of each field. +If `FALSE`, the pattern is treated as a simple string. -If FALSE, the function returns the name and value of each field. +*Default:* `TRUE` +| Boolean + +| **patternspace** + +__optional__ +| A string literal with two possible values. + +`"field"`: The pattern is matched against individual field names. + +`"path"`: The pattern is matched against composite path names. + +*Default:* `"path"` +| String + +| **exact** + +__optional__ +| Determines whether the provided pattern must be an exact match for the field or path (as determined by the `patternspace` parameter). + +This is a short-cut for the regular expression start (^) and end ($) anchors, and can be used even when `regex` is set to `false`. + +If `TRUE`, the pattern must be an exact match. + +If `FALSE`, the pattern does not need to be an exact match. + +*Default:* `FALSE` +| Boolean + +| **ignorecase** + +__optional__ +| If `TRUE`, the pattern matching is case-sensitive. + +If `FALSE`, the pattern matching ignores case. *Default:* `FALSE` | Boolean + |=== === Return Value @@ -1021,7 +1073,9 @@ WITH input AS ({ SELECT OBJECT_PAIRS_NESTED(input) AS nested_pairs, OBJECT_PAIRS_NESTED(input, {"composites": true}) AS nested_pairs_comp, OBJECT_PAIRS_NESTED(input, {"pattern": "name"}) AS nested_pairs_pattern, - OBJECT_PAIRS_NESTED(input, {"types": true}) AS nested_pairs_types; + OBJECT_PAIRS_NESTED(input, {"types": true}) AS nested_pairs_types, + OBJECT_PAIRS_NESTED (input, {"pattern": "name", "patternspace":"field"}) as filtered_fields, + OBJECT_PAIRS_NESTED (input, {"pattern": "-name$", "patternspace":"field", "regex":true}) as filtered_fields_regex; ---- .Results @@ -1071,6 +1125,18 @@ SELECT OBJECT_PAIRS_NESTED(input) AS nested_pairs, "name": "attribute.flight-number", "type": "number" } + ], + "filtered_fields": [ + { + "name": "attribute.flight-name", + "val": "AI444" + } + ], + "filtered_fields_regex": [ + { + "name": "attribute.flight-name", + "val": "AI444" + } ] } ] @@ -1224,6 +1290,11 @@ Default `"path"`. `"path"`::: The pattern is matched against composite path names. +exact;; A boolean. +Specifies whether the provided pattern must be an exact match for the field or path (determined by the `patternspace` parameter). +It is a short-cut for the regular expression start (^) and end ($) anchors, and can be used even when `regex` is set to `false`. +Default `false`. + === Return Value An array containing the full path to every possible field within the source object, subject to the specified options. From 6b18e36ca39b484981bd56340b37fad546fac12f Mon Sep 17 00:00:00 2001 From: Rakhi Prathap Date: Thu, 10 Jul 2025 14:12:48 +0530 Subject: [PATCH 5/6] add additional fields --- .../n1ql-language-reference/objectfun.adoc | 139 ++++++++++++++---- 1 file changed, 112 insertions(+), 27 deletions(-) diff --git a/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc b/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc index 2b749130d..33616540a 100644 --- a/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc @@ -301,11 +301,14 @@ options:: [Optional] A JSON object specifying options for the function. | Name | Description | Schema | **pattern** + +__optional__ + | The pattern to match. This can be a regular expression or a simple string, depending on the `regex` parameter. | String | **regex** + +__optional__ | If `TRUE`, the pattern is treated as a regular expression. If `FALSE`, the pattern is treated as a simple string. @@ -314,6 +317,7 @@ If `FALSE`, the pattern is treated as a simple string. | Boolean | **arraysubscript** + +__optional__ | Specifies whether array subscripts are included in field names before applying the filter. If `TRUE`, array subscripts are included. @@ -325,6 +329,7 @@ If `FALSE`, array subscripts are replaced by `*`. | Boolean | **composites** + +__optional__ | Specifies whether the pattern should match field names that contain values requiring further processing, such as nested objects or arrays. If `TRUE`, the pattern is matched against every level of nested fields. @@ -335,6 +340,7 @@ If `FALSE`, the pattern is matched only against the deepest level of nested fiel | Boolean | **patternspace** + +__optional__ | A string literal with two possible values. `"field"`: The pattern is matched against individual field names. @@ -345,9 +351,23 @@ If `FALSE`, the pattern is matched only against the deepest level of nested fiel | String | **exact** + -| Specifies whether the provided pattern must be an exact match for the field or path (determined by the `patternspace` parameter). +__optional__ +| Determines whether the provided pattern must be an exact match for the field or path (as determined by the `patternspace` parameter). + +This is a short-cut for the regular expression start (^) and end ($) anchors, and can be used even when `regex` is set to `false`. + +If `TRUE`, the pattern must be an exact match. -It is a short-cut for the regular expression start (^) and end ($) anchors, and you can use it even when `regex` is set to `false`. +If `FALSE`, the pattern does not need to be an exact match. + +*Default:* `FALSE` +| Boolean + +| **ignorecase** + +__optional__ +| If `TRUE`, the pattern matching is case-sensitive. + +If `FALSE`, the pattern matching ignores case. *Default:* `FALSE` | Boolean @@ -1029,6 +1049,20 @@ If `FALSE`, the pattern matching ignores case. *Default:* `FALSE` | Boolean +| **report** + +__optional__ + +| Controls the output of the `name` field, allowing selection between the full path or the final field name. + +Possible values are: + +`"field"`: Outputs only the final field name. + +`"path"`: Outputs the full path to the field. + +*Default:* `"path"` + +| String |=== === Return Value @@ -1074,8 +1108,9 @@ SELECT OBJECT_PAIRS_NESTED(input) AS nested_pairs, OBJECT_PAIRS_NESTED(input, {"composites": true}) AS nested_pairs_comp, OBJECT_PAIRS_NESTED(input, {"pattern": "name"}) AS nested_pairs_pattern, OBJECT_PAIRS_NESTED(input, {"types": true}) AS nested_pairs_types, - OBJECT_PAIRS_NESTED (input, {"pattern": "name", "patternspace":"field"}) as filtered_fields, - OBJECT_PAIRS_NESTED (input, {"pattern": "-name$", "patternspace":"field", "regex":true}) as filtered_fields_regex; + OBJECT_PAIRS_NESTED(input, {"pattern": "number", "patternspace":"field"}) AS filtered_fields, + OBJECT_PAIRS_NESTED(input, {"pattern": "-name$", "patternspace":"field", "regex":true}) + AS filtered_fields_regex; ---- .Results @@ -1128,8 +1163,8 @@ SELECT OBJECT_PAIRS_NESTED(input) AS nested_pairs, ], "filtered_fields": [ { - "name": "attribute.flight-name", - "val": "AI444" + "name": "attribute.flight-number", + "val": 737 } ], "filtered_fields_regex": [ @@ -1264,36 +1299,86 @@ A field in this context may be any attribute or element, nested at any level wit object:: An expression representing an object. -options:: [Optional] An object containing the following possible parameters: +options:: [Optional] An object containing the following possible parameters. + +=== Options + +[options="header", cols="1a,3a,1a"] +|=== +| Name | Description | Schema + +| **composites** + +__optional__ +| If `TRUE`, every level of every nested field is displayed. + +If `FALSE`, only the deepest possible nested fields are returned. + +*Default:* `TRUE` +| Boolean + +| **arraysubscript** + +__optional__ +| If `TRUE`, array subscripts are returned. + +If `FALSE`, array subscripts are replaced by `*`. + +*Default:* `TRUE` +| Boolean -composites;; A boolean. -If `true`, every level of every nested field is displayed; if `false`, only the deepest possible nested fields are returned. -Default `true`. +| **unique** + +__optional__ +| If `TRUE`, dduplicate field names are collapsed to single unique field name. -arraysubscript;; A boolean. -If `true`, array subscripts are returned; if `false`, array subscripts are replaced by `*`. -Default `true`. +If `FALSE`, all duplicate field names are returned. -unique;; A boolean. -If `true`, duplicate field names are collapsed to single unique field name; if `false`, all duplicate field names are returned. Typically used when arrays are expanded and array subscripts are not returned. -Default `true`. -pattern;; A regular expression used to filter the returned paths. +*Default:* `TRUE` +| Boolean + +| **pattern** + +__optional__ +| A regular expression used to filter the returned paths. Used in conjunction with the following setting. -patternspace;; A string literal with two possible values. -Default `"path"`. -+ -[horizontal] -`"field"`::: The pattern is matched against individual field names. +| String + +| **patternspace** + +__optional__ + +| A string literal with two possible values. + +`"field"`: The pattern is matched against individual field names. + +`"path"`: The pattern is matched against composite path names. + +*Default:* `"path"` -`"path"`::: The pattern is matched against composite path names. +| String + +| **exact** + +__optional__ +| Determines whether the provided pattern must be an exact match for the field or path (as determined by the `patternspace` parameter). + +This is a short-cut for the regular expression start (^) and end ($) anchors, and can be used even when `regex` is set to `false`. + +If `TRUE`, the pattern must be an exact match. + +If `FALSE`, the pattern does not need to be an exact match. + +*Default:* `FALSE` +| Boolean + +| **ignorecase** + +__optional__ +| If `TRUE`, the pattern matching is case-sensitive. -exact;; A boolean. -Specifies whether the provided pattern must be an exact match for the field or path (determined by the `patternspace` parameter). -It is a short-cut for the regular expression start (^) and end ($) anchors, and can be used even when `regex` is set to `false`. -Default `false`. +If `FALSE`, the pattern matching ignores case. + +*Default:* `FALSE` +| Boolean + +|=== === Return Value From 666352050f84e10db0c52271f3169ab8dd9aa997 Mon Sep 17 00:00:00 2001 From: Rakhi Prathap Date: Fri, 11 Jul 2025 10:00:35 +0530 Subject: [PATCH 6/6] Minor changes --- .../n1ql-language-reference/objectfun.adoc | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc b/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc index 33616540a..108f90822 100644 --- a/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/objectfun.adoc @@ -352,7 +352,7 @@ __optional__ | **exact** + __optional__ -| Determines whether the provided pattern must be an exact match for the field or path (as determined by the `patternspace` parameter). +| Specifies whether the provided pattern must be an exact match for the field or path (as determined by the `patternspace` parameter). This is a short-cut for the regular expression start (^) and end ($) anchors, and can be used even when `regex` is set to `false`. @@ -1327,7 +1327,7 @@ If `FALSE`, array subscripts are replaced by `*`. | **unique** + __optional__ -| If `TRUE`, dduplicate field names are collapsed to single unique field name. +| If `TRUE`, duplicate field names are collapsed to single unique field name. If `FALSE`, all duplicate field names are returned. @@ -1338,8 +1338,8 @@ Typically used when arrays are expanded and array subscripts are not returned. | **pattern** + __optional__ -| A regular expression used to filter the returned paths. -Used in conjunction with the following setting. +| A regular expression used to filter the returned paths. +Used in conjunction with the `patternspace` setting. | String @@ -1470,7 +1470,7 @@ SELECT [[obj-paths-ex3,OBJECT_PATHS() Example 3]] .Pattern matching and pattern space ==== -This example searches for strings beginning with "n" in the given object paths. +This example searches for strings beginning with "n" and also fields that exactly match "name". .Query [source,sqlpp] @@ -1484,7 +1484,9 @@ SELECT OBJECT_PATHS(input, {"pattern": "^n", "patternspace": "field"}) AS field_starts_with_n, OBJECT_PATHS(input, {"pattern": "^n", "patternspace": "path"}) - AS path_starts_with_n; + AS path_starts_with_n, + OBJECT_PATHS(input, {"pattern": "name", "patternspace": "field", "exact": true}) + AS exact_field_name; ---- .Results @@ -1499,7 +1501,10 @@ SELECT "field_starts_with_n": [ "attribute.name" ], - "path_starts_with_n": [] + "path_starts_with_n": [], + "exact_field_name": [ + "attribute.name" + ] } ] ----