Merged PR 1594069: [engine][collation] Support for find

### Does this PR have any customer impact? No

### Type (Feature, Refactoring, Bugfix, DevOps, Testing, Perf, etc) Feature

### Does it involve schema level changes? (Table, Column, Index, UDF, etc level changes) Yes

### Are you introducing any new config? If yes, do you have tests with and without them being set? No

### ChangeLog (Refer [Template](../oss/CHANGELOG.md))

### Description
Support collation with `find` queries.

----
#### AI description  (iteration 1)
#### PR Classification
New feature

#### PR Summary
This pull request adds support for the `project find` functionality with collation in the engine.
- Added new SQL functions in `sql/udfs/projection/bson_projection--0.102-0.sql` for handling BSON projection with collation.
- Updated `metadata_cache.c` to include OID caching for the new `bson_dollar_project_find` function with collation.
- Modified `bson_project.c` to handle additional arguments for collation in the `bson_dollar_project_find` function.
- Updated `bson_projection--latest.sql` to include the new function definition.
- Added function declaration in `metadata_cache.h` for the new `bson_dollar_project_find` function with collation.
<!-- GitOpsUserAgent=GitOps.Apps.Server.pullrequestcopilot -->

Related work items: #3924476

CHANGELOG.md

@@ -13,6 +13,7 @@
 * Support collation with `$in` array operator *[Feature]*
 * Support unique index truncation by default with new operator class *[Feature]*
 * Support collation in aggregation `$redact` stage *[Feature]*
+* Support collation in aggregation `find` *[Feature]*
 * Support collation in aggregation `$set` stage *[Feature]*
 * Support collation in aggregation `$addFields` stage *[Feature]*
 * Support collation with `$setEquals`, `$setUnion`, `$setIntersection`, `$setDifference`, `$setIsSubet` in the aggregation pipeline *[Feature]*

internal/pg_documentdb_distributed/src/test/regress/expected/bson_dollar_ops_collation_tests_runtime.out

@@ -2263,6 +2263,57 @@ SELECT document FROM bson_aggregation_pipeline('db', '{ "aggregate": "coll_agg_p
  { "a" : "bat", "newField" : false }
 (6 rows)
 
+-- find
+SELECT document FROM bson_aggregation_find('db', '{ "find": "coll_agg_proj", "projection": { "a": 1, "newField": { "$eq": ["$a", "CAT"] } }, "sort": { "_id": 1 }, "skip": 0, "limit": 5, "collation": { "locale": "en", "strength" : 1} }');
+                              document                               
+---------------------------------------------------------------------
+ { "_id" : { "$numberInt" : "1" }, "a" : "cat", "newField" : true }
+ { "_id" : { "$numberInt" : "2" }, "a" : "dog", "newField" : false }
+ { "_id" : { "$numberInt" : "3" }, "a" : "cAt", "newField" : true }
+ { "_id" : { "$numberInt" : "4" }, "a" : "dOg", "newField" : false }
+ { "_id" : "bat", "a" : "bat", "newField" : false }
+(5 rows)
+
+SELECT document FROM bson_aggregation_find('db', '{ "find": "coll_agg_proj", "projection": { "a": 1, "newField": { "$ne": ["$a", "CAT"] } }, "sort": { "_id": 1 }, "skip": 0, "limit": 5, "collation": { "locale": "en", "strength" : 2} }');
+                              document                               
+---------------------------------------------------------------------
+ { "_id" : { "$numberInt" : "1" }, "a" : "cat", "newField" : false }
+ { "_id" : { "$numberInt" : "2" }, "a" : "dog", "newField" : true }
+ { "_id" : { "$numberInt" : "3" }, "a" : "cAt", "newField" : false }
+ { "_id" : { "$numberInt" : "4" }, "a" : "dOg", "newField" : true }
+ { "_id" : "bat", "a" : "bat", "newField" : true }
+(5 rows)
+
+SELECT document FROM bson_aggregation_find('db', '{ "find": "coll_agg_proj", "projection": { "a": 1, "newField": { "$ne": ["$a", "CAT"] } }, "sort": { "_id": 1 }, "skip": 0, "limit": 5, "collation": { "locale": "en", "strength" : 3} }');
+                              document                              
+---------------------------------------------------------------------
+ { "_id" : { "$numberInt" : "1" }, "a" : "cat", "newField" : true }
+ { "_id" : { "$numberInt" : "2" }, "a" : "dog", "newField" : true }
+ { "_id" : { "$numberInt" : "3" }, "a" : "cAt", "newField" : true }
+ { "_id" : { "$numberInt" : "4" }, "a" : "dOg", "newField" : true }
+ { "_id" : "bat", "a" : "bat", "newField" : true }
+(5 rows)
+
+SELECT document FROM bson_aggregation_find('db', '{ "find": "coll_agg_proj", "projection": { "a": 1, "newField": { "$gte": ["$a", "CAT"] } }, "sort": { "_id": 1 }, "skip": 0, "limit": 5, "collation": { "locale": "en", "strength" : 1} }');
+                              document                              
+---------------------------------------------------------------------
+ { "_id" : { "$numberInt" : "1" }, "a" : "cat", "newField" : true }
+ { "_id" : { "$numberInt" : "2" }, "a" : "dog", "newField" : true }
+ { "_id" : { "$numberInt" : "3" }, "a" : "cAt", "newField" : true }
+ { "_id" : { "$numberInt" : "4" }, "a" : "dOg", "newField" : true }
+ { "_id" : "bat", "a" : "bat", "newField" : false }
+(5 rows)
+
+SELECT document FROM bson_aggregation_find('db', '{ "find": "coll_agg_proj", "projection": { "a": 1, "newField": { "$gte": ["$a", "CAT"] } }, "sort": { "_id": 1 }, "skip": 0, "limit": 5, "collation": { "locale": "en", "strength" : 3} }');
+                              document                               
+---------------------------------------------------------------------
+ { "_id" : { "$numberInt" : "1" }, "a" : "cat", "newField" : false }
+ { "_id" : { "$numberInt" : "2" }, "a" : "dog", "newField" : true }
+ { "_id" : { "$numberInt" : "3" }, "a" : "cAt", "newField" : false }
+ { "_id" : { "$numberInt" : "4" }, "a" : "dOg", "newField" : true }
+ { "_id" : "bat", "a" : "bat", "newField" : false }
+(5 rows)
+
 -- redact
 SELECT documentdb_api.insert_one('db','coll_redact','{ "_id": 1, "level": "public", "content": "content 1", "details": { "level": "public", "value": "content 1.1", "moreDetails": { "level": "restricted", "info": "content 1.1.1" } } }', NULL);
 NOTICE:  creating collection

internal/pg_documentdb_distributed/src/test/regress/sql/bson_dollar_ops_collation_tests_runtime.sql

@@ -522,6 +522,13 @@ SELECT document FROM bson_aggregation_pipeline('db', '{ "aggregate": "coll_agg_p
 SELECT document FROM bson_aggregation_pipeline('db', '{ "aggregate": "coll_agg_proj", "pipeline": [ { "$replaceRoot": { "newRoot": { "a": "$a", "newField": { "$lte": ["$a", "DoG"] } } } } ], "cursor": {}, "collation": { "locale": "fr", "strength" : 1} }');
 SELECT document FROM bson_aggregation_pipeline('db', '{ "aggregate": "coll_agg_proj", "pipeline": [ { "$replaceRoot": { "newRoot": { "a": "$a", "newField": { "$gte": ["$a", "doG"] } } } } ], "cursor": {}, "collation": { "locale": "en", "strength" : 3} }');
 
+-- find
+SELECT document FROM bson_aggregation_find('db', '{ "find": "coll_agg_proj", "projection": { "a": 1, "newField": { "$eq": ["$a", "CAT"] } }, "sort": { "_id": 1 }, "skip": 0, "limit": 5, "collation": { "locale": "en", "strength" : 1} }');
+SELECT document FROM bson_aggregation_find('db', '{ "find": "coll_agg_proj", "projection": { "a": 1, "newField": { "$ne": ["$a", "CAT"] } }, "sort": { "_id": 1 }, "skip": 0, "limit": 5, "collation": { "locale": "en", "strength" : 2} }');
+SELECT document FROM bson_aggregation_find('db', '{ "find": "coll_agg_proj", "projection": { "a": 1, "newField": { "$ne": ["$a", "CAT"] } }, "sort": { "_id": 1 }, "skip": 0, "limit": 5, "collation": { "locale": "en", "strength" : 3} }');
+SELECT document FROM bson_aggregation_find('db', '{ "find": "coll_agg_proj", "projection": { "a": 1, "newField": { "$gte": ["$a", "CAT"] } }, "sort": { "_id": 1 }, "skip": 0, "limit": 5, "collation": { "locale": "en", "strength" : 1} }');
+SELECT document FROM bson_aggregation_find('db', '{ "find": "coll_agg_proj", "projection": { "a": 1, "newField": { "$gte": ["$a", "CAT"] } }, "sort": { "_id": 1 }, "skip": 0, "limit": 5, "collation": { "locale": "en", "strength" : 3} }');
+
 -- redact
 SELECT documentdb_api.insert_one('db','coll_redact','{ "_id": 1, "level": "public", "content": "content 1", "details": { "level": "public", "value": "content 1.1", "moreDetails": { "level": "restricted", "info": "content 1.1.1" } } }', NULL);
 SELECT documentdb_api.insert_one('db','coll_redact','{ "_id": 2, "level": "restricted", "content": "content 2", "details": { "level": "public", "value": "content 2.1", "moreDetails": { "level": "restricted", "info": "content 2.1.1" } } }', NULL);

pg_documentdb/include/metadata/metadata_cache.h

@@ -299,6 +299,7 @@ Oid BsonDollarMergeExtractFilterFunctionOid(void);
 Oid BsonDollarMergeJoinFunctionOid(void);
 Oid BsonDollarProjectFindFunctionOid(void);
 Oid BsonDollarProjectFindWithLetFunctionOid(void);
+Oid BsonDollarProjectFindWithLetAndCollationFunctionOid(void);
 Oid BsonDollarUnwindFunctionOid(void);
 Oid BsonDollarUnwindWithOptionsFunctionOid(void);
 Oid BsonDollarReplaceRootFunctionOid(void);

pg_documentdb/sql/udfs/projection/bson_projection--0.102-0.sql

@@ -29,6 +29,12 @@ CREATE OR REPLACE FUNCTION __API_SCHEMA_INTERNAL_V2__.bson_dollar_project_find(d
  IMMUTABLE PARALLEL SAFE CALLED ON NULL INPUT
 AS 'MODULE_PATHNAME', $function$bson_dollar_project_find$function$;
 
+CREATE OR REPLACE FUNCTION __API_SCHEMA_INTERNAL_V2__.bson_dollar_project_find(document __CORE_SCHEMA_V2__.bson, pathSpec __CORE_SCHEMA_V2__.bson, querySpec __CORE_SCHEMA_V2__.bson, letVariableSpec __CORE_SCHEMA_V2__.bson, collationString text)
+ RETURNS __CORE_SCHEMA_V2__.bson
+ LANGUAGE c
+ IMMUTABLE PARALLEL SAFE CALLED ON NULL INPUT
+AS 'MODULE_PATHNAME', $function$bson_dollar_project_find$function$;
+
 CREATE OR REPLACE FUNCTION __API_CATALOG_SCHEMA__.bson_dollar_add_fields(document __CORE_SCHEMA__.bson, pathSpec __CORE_SCHEMA__.bson)
  RETURNS __CORE_SCHEMA__.bson
  LANGUAGE c

pg_documentdb/sql/udfs/projection/bson_projection--latest.sql

@@ -29,6 +29,12 @@ CREATE OR REPLACE FUNCTION __API_SCHEMA_INTERNAL_V2__.bson_dollar_project_find(d
  IMMUTABLE PARALLEL SAFE CALLED ON NULL INPUT
 AS 'MODULE_PATHNAME', $function$bson_dollar_project_find$function$;
 
+CREATE OR REPLACE FUNCTION __API_SCHEMA_INTERNAL_V2__.bson_dollar_project_find(document __CORE_SCHEMA_V2__.bson, pathSpec __CORE_SCHEMA_V2__.bson, querySpec __CORE_SCHEMA_V2__.bson, letVariableSpec __CORE_SCHEMA_V2__.bson, collationString text)
+ RETURNS __CORE_SCHEMA_V2__.bson
+ LANGUAGE c
+ IMMUTABLE PARALLEL SAFE CALLED ON NULL INPUT
+AS 'MODULE_PATHNAME', $function$bson_dollar_project_find$function$;
+
 CREATE OR REPLACE FUNCTION __API_CATALOG_SCHEMA__.bson_dollar_add_fields(document __CORE_SCHEMA__.bson, pathSpec __CORE_SCHEMA__.bson)
  RETURNS __CORE_SCHEMA__.bson
  LANGUAGE c

pg_documentdb/src/aggregation/bson_aggregation_pipeline.c

@@ -3109,7 +3109,19 @@ HandleProjectFind(const bson_value_t *existingValue, const bson_value_t *queryVa
 
 	List *args;
 	Oid funcOid = BsonDollarProjectFindFunctionOid();
-	if (context->variableSpec != NULL)
+	if (IsCollationApplicable(context->collationString) && IsClusterVersionAtleast(
+			DocDB_V0, 102, 0))
+	{
+		pgbson *queryDoc = queryValue->value_type == BSON_TYPE_EOD ? PgbsonInitEmpty() :
+						   PgbsonInitFromDocumentBsonValue(queryValue);
+		Const *collationStringConst = MakeTextConst(context->collationString,
+													strlen(context->collationString));
+
+		args = list_make5(currentProjection, projectProcessed, MakeBsonConst(queryDoc),
+						  context->variableSpec, collationStringConst);
+		funcOid = BsonDollarProjectFindWithLetAndCollationFunctionOid();
+	}
+	else if (context->variableSpec != NULL)
 	{
 		pgbson *queryDoc = queryValue->value_type == BSON_TYPE_EOD ? PgbsonInitEmpty() :
 						   PgbsonInitFromDocumentBsonValue(queryValue);
@@ -3128,7 +3140,6 @@ HandleProjectFind(const bson_value_t *existingValue, const bson_value_t *queryVa
 		args = list_make3(currentProjection, projectProcessed, queryDocProcessed);
 	}
 
-
 	FuncExpr *resultExpr = makeFuncExpr(funcOid, BsonTypeId(), args, InvalidOid,
 										InvalidOid, COERCE_EXPLICIT_CALL);
 

pg_documentdb/src/aggregation/bson_project.c

@@ -405,6 +405,7 @@ bson_dollar_project_find(PG_FUNCTION_ARGS)
 	pgbson *pathSpec = PG_GETARG_PGBSON(1);
 	pgbson *querySpec = NULL;
 	pgbson *variableSpec = NULL;
+	char *collationString = NULL;
 
 	if (PG_NARGS() > 2)
 	{
@@ -424,7 +425,7 @@ bson_dollar_project_find(PG_FUNCTION_ARGS)
 
 	const BsonProjectionQueryState *state;
 
-	int argPosition[2] = { 1, 0 };
+	int argPosition[3] = { 1, 0, 0 };
 	int numArgs = 1;
 
 	if (PG_NARGS() > 3)
@@ -435,6 +436,14 @@ bson_dollar_project_find(PG_FUNCTION_ARGS)
 		numArgs = 2;
 	}
 
+	if (EnableCollation && PG_NARGS() > 4)
+	{
+		collationString = PG_ARGISNULL(4) ? NULL : text_to_cstring(PG_GETARG_TEXT_P(4));
+
+		argPosition[2] = 4;
+		numArgs = 3;
+	}
+
 	bson_iter_t pathSpecIter;
 	PgbsonInitIterator(pathSpec, &pathSpecIter);
 
@@ -443,7 +452,8 @@ bson_dollar_project_find(PG_FUNCTION_ARGS)
 		.allowInclusionExclusion = false,
 		.pathSpecIter = &pathSpecIter,
 		.querySpec = querySpec,
-		.variableSpec = variableSpec
+		.variableSpec = variableSpec,
+		.collationString = collationString
 	};
 
 	SetCachedFunctionStateMultiArgs(

pg_documentdb/src/metadata/metadata_cache.c

@@ -687,6 +687,9 @@ typedef struct DocumentDBApiOidCacheData
 	/* OID of the bson_dollar_project_find with let args function */
 	Oid ApiCatalogBsonDollarProjectFindWithLetFunctionOid;
 
+	/* OID of the bson_dollar_project_find with let and collation function */
+	Oid ApiCatalogBsonDollarProjectFindWithLetAndCollationFunctionOid;
+
 	/* OID of the bson_dollar_unwind(bson, text) function */
 	Oid ApiCatalogBsonDollarUnwindFunctionOid;
 
@@ -3300,6 +3303,20 @@ BsonDollarProjectFindWithLetFunctionOid(void)
 }
 
 
+Oid
+BsonDollarProjectFindWithLetAndCollationFunctionOid(void)
+{
+	return GetOperatorFunctionIdFiveArgs(
+		&Cache.ApiCatalogBsonDollarProjectFindWithLetAndCollationFunctionOid,
+		DocumentDBApiInternalSchemaName,
+		"bson_dollar_project_find",
+		DocumentDBCoreBsonTypeId(), DocumentDBCoreBsonTypeId(),
+		DocumentDBCoreBsonTypeId(),
+		DocumentDBCoreBsonTypeId(),
+		TEXTOID);
+}
+
+
 Oid
 BsonDollarMergeHandleWhenMatchedFunctionOid(void)
 {

pg_documentdb/src/test/regress/expected/public_api_schema.out

@@ -225,6 +225,7 @@
  documentdb_api_internal | bson_dollar_project                          | documentdb_core.bson       | document documentdb_core.bson, pathspec documentdb_core.bson, variablespec documentdb_core.bson                                                                                                                                                                                                                                                                                                                                                                                                                                                 | func
  documentdb_api_internal | bson_dollar_project                          | documentdb_core.bson       | document documentdb_core.bson, pathspec documentdb_core.bson, variablespec documentdb_core.bson, collationstring text                                                                                                                                                                                                                                                                                                                                                                                                                           | func
  documentdb_api_internal | bson_dollar_project_find                     | documentdb_core.bson       | document documentdb_core.bson, pathspec documentdb_core.bson, queryspec documentdb_core.bson, letvariablespec documentdb_core.bson                                                                                                                                                                                                                                                                                                                                                                                                              | func
+ documentdb_api_internal | bson_dollar_project_find                     | documentdb_core.bson       | document documentdb_core.bson, pathspec documentdb_core.bson, queryspec documentdb_core.bson, letvariablespec documentdb_core.bson, collationstring text                                                                                                                                                                                                                                                                                                                                                                                        | func
  documentdb_api_internal | bson_dollar_range                            | boolean                    | documentdb_core.bson, documentdb_core.bson                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | func
  documentdb_api_internal | bson_dollar_redact                           | documentdb_core.bson       | documentdb_core.bson, documentdb_core.bson, text, documentdb_core.bson                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | func
  documentdb_api_internal | bson_dollar_redact                           | documentdb_core.bson       | document documentdb_core.bson, redactspec documentdb_core.bson, redactspectext text, variablespec documentdb_core.bson, collationstring text                                                                                                                                                                                                                                                                                                                                                                                                    | func
@@ -381,7 +382,7 @@
  documentdb_api_internal | update_one                                   | record                     | p_collection_id bigint, p_shard_key_value bigint, p_query documentdb_core.bson, p_update documentdb_core.bson, p_shard_key documentdb_core.bson, p_is_upsert boolean, p_sort documentdb_core.bson, p_return_old_or_new boolean, p_return_fields documentdb_core.bson, p_array_filters documentdb_core.bson, p_transaction_id text, OUT o_is_row_updated boolean, OUT o_update_skipped boolean, OUT o_is_retry boolean, OUT o_reinsert_document documentdb_core.bson, OUT o_upserted_object_id bytea, OUT o_result_document documentdb_core.bson | func
  documentdb_api_internal | update_worker                                | documentdb_core.bson       | p_collection_id bigint, p_shard_key_value bigint, p_shard_oid regclass, p_update_internal_spec documentdb_core.bson, p_update_internal_docs documentdb_core.bsonsequence, p_transaction_id text                                                                                                                                                                                                                                                                                                                                                 | func
  documentdb_api_internal | validate_dbname                              | void                       | dbname text                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | func
-(206 rows)
+(207 rows)
 
 \df documentdb_data.*
                        List of functions