[SPARK-56714] [SQL] Remove __metadata_col metadata from aggregated access only columns

[mihailoale-db]

What changes were proposed in this pull request?

In this PR I propose to remove __metadata_col metadata from aggregated access only columns. It's not semantically needed and it blocks metadata column resolution work in single-pass resolver.

Why are the changes needed?

To make aggregated access only metadata semantically sound.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Existing tests.

Was this patch authored or co-authored using generative AI tooling?

No.

[cloud-fan]

Nice cleanup — the prior coupling looked accidental. I traced the only setter call site (NameScope.updateHiddenOutputProperties) and the only predicate call site (NameScope.getHiddenOutputCandidates); neither depends on aggregatedAccessOnly attributes also being metadata columns, so decoupling is sound and existing behavior is preserved.

Two small, optional follow-ups:

1. The doc on AGGREGATED_ACCESS_ONLY (lines 179–185) still says "If set, this metadata column can only be accessed under AggregateExpression." After this PR the marker is not restricted to metadata columns — could you tweak the wording (e.g. "this attribute" / "this column") so the comment matches the new contract?

2. There is a symmetric unit test for the analogous metadata key — AnalysisSuite.scala:1761 "SPARK-43293: __qualified_access_only should be ignored in normal columns". An equivalent targeted test for __aggregated_access_only (e.g. that a non-metadata attribute marked via markAsAggregatedAccessOnly() does not satisfy isMetadataCol) would lock the new contract in directly, on top of the existing end-to-end SQL coverage.

[mihailoale-db]

Nice cleanup — the prior coupling looked accidental. I traced the only setter call site (NameScope.updateHiddenOutputProperties) and the only predicate call site (NameScope.getHiddenOutputCandidates); neither depends on aggregatedAccessOnly attributes also being metadata columns, so decoupling is sound and existing behavior is preserved.

Two small, optional follow-ups:

1. The doc on AGGREGATED_ACCESS_ONLY (lines 179–185) still says "If set, this metadata column can only be accessed under AggregateExpression." After this PR the marker is not restricted to metadata columns — could you tweak the wording (e.g. "this attribute" / "this column") so the comment matches the new contract?
2. There is a symmetric unit test for the analogous metadata key — AnalysisSuite.scala:1761 "SPARK-43293: __qualified_access_only should be ignored in normal columns". An equivalent targeted test for __aggregated_access_only (e.g. that a non-metadata attribute marked via markAsAggregatedAccessOnly() does not satisfy isMetadataCol) would lock the new contract in directly, on top of the existing end-to-end SQL coverage.

Addressed your comments. PTAL again.

[cloud-fan]

the proto failure is unrelated, thanks, merging to master/4.x/4.2!