[BugFix] Return all columns (struct and nested fields) listed when using head

[quangdutran]

Description

When both a struct parent and the nested ones are projected in the query, appending | head must not drop the nested.

Explanation

Query with head has a Project wrapper like Project(Head (Project([agent.id, agent.name, agent])))

// source=big5 | fields agent.id agent.name agent | head

Seems like outer Project hits tryToRemoveNestedFields so nested fields are removed. Without head, there is no wrapper, hence nested fields stay.

tryToRemoveMetaFields has one run condition: There is no other project ever visited in the main query
So I try to apply the same with tryToRemoveNestedFields

Related Issues

Resolves #5507

Check List

• New functionality includes testing.
• New functionality has been documented.
• New functionality has javadoc added.
• New functionality has a user manual doc added.
• New PPL command checklist all confirmed.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff or -s.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[quangdutran]

Test result:

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 PR contains tests

🔒 No security concerns identified

✅ No TODO sections

🔀 No multiple PR themes

⚡ Recommended focus areas for review Typo in Method Name Method name testStruckFieldAndSubFieldWithHead contains a typo. 'Struck' should be 'Struct' to accurately describe testing struct fields. This affects code readability and searchability. public void testStruckFieldAndSubFieldWithHead() throws IOException { Missing Data Verification The test verifies schema but does not verify actual data rows returned by the query. Without data verification, the test cannot confirm that both the struct parent and nested fields are correctly returned in the result set, which is the core bug being fixed. public void testStruckFieldAndSubFieldWithHead() throws IOException { JSONObject result = executeQuery( String.format("source=%s | fields city.name, city | head", TEST_INDEX_DEEP_NESTED)); verifySchema(result, schema("city.name", "string"), schema("city", "struct")); }

[github-actions]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Fix typo in test method name The test method name contains a typo: "Struck" should be "Struct". This typo could cause confusion when reading test reports or searching for struct-related tests. Rename the method to testStructFieldAndSubFieldWithHead for clarity. integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalciteEvalCommandIT.java [239-244] @Test -public void testStruckFieldAndSubFieldWithHead() throws IOException { +public void testStructFieldAndSubFieldWithHead() throws IOException { JSONObject result = executeQuery( String.format("source=%s | fields city.name, city | head", TEST_INDEX_DEEP_NESTED)); verifySchema(result, schema("city.name", "string"), schema("city", "struct")); } Suggestion importance[1-10]: 5 __ Why: The suggestion correctly identifies a typo in the method name testStruckFieldAndSubFieldWithHead where "Struck" should be "Struct". This improves code readability and maintainability, though it's a minor issue that doesn't affect functionality.	Low
	Add data row verification to test The test only verifies the schema but doesn't verify the actual data rows returned. Add verifyDataRows to ensure the query returns expected data, not just the correct schema structure. This would make the test more comprehensive and catch potential data retrieval issues. integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalciteEvalCommandIT.java [239-244] @Test public void testStruckFieldAndSubFieldWithHead() throws IOException { JSONObject result = executeQuery( String.format("source=%s | fields city.name, city | head", TEST_INDEX_DEEP_NESTED)); verifySchema(result, schema("city.name", "string"), schema("city", "struct")); + verifyDataRows(result); } Suggestion importance[1-10]: 3 __ Why: While adding verifyDataRows could make the test more comprehensive, the suggestion provides an incomplete implementation (calling verifyDataRows(result) without expected data rows). The current test may intentionally focus only on schema verification, making this a marginal improvement without proper implementation details.	Low