Spark3: Added segments & grammar needed for hints

[R7L208]

Brief summary of the change made

• Added test cases for Hints in a SELECT statement
• Added segments and grammar to distinguish hints in Spark3 from general comments
• Included additional detail from Ansi docstring in Spark3 for UnorderedSelectStatementSegment

Are there any other side effects of this change that we should be aware of?

N/A

Pull Request checklist

• Please confirm you have completed any of the necessary steps below.

• Included test cases to demonstrate any code changes, which may be one or more of the following:

  • .yml rule test cases in test/fixtures/rules/std_rule_cases.
  • .sql/.yml parser test cases in test/fixtures/dialects (note YML files can be auto generated with tox -e generate-fixture-yml).
  • Full autofix test cases in test/fixtures/linter/autofix.
  • Other.

• Added appropriate documentation for the change.

• Created GitHub issues for any relevant followup/future enhancements if appropriate.

[codecov]

Codecov Report

Merging #2528 (8cc7fb0) into main (986e66f) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##              main     #2528   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          162       162           
  Lines        11720     11734   +14     
=========================================
+ Hits         11720     11734   +14     

Impacted Files	Coverage Δ
src/sqlfluff/dialects/dialect_spark3.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 986e66f...8cc7fb0. Read the comment docs.

[R7L208]

I'm about to push most recent changes but am looking at L027.PY.

Everything is parsing but the table reference inside of the hint is being incorrectly flagged by the rule.

-- Join Hints for broadcast join
SELECT /*+ BROADCAST(t1) */
    t1.a,
    t1.b,
    t2.c
FROM t1 INNER JOIN t2 ON t1.key = t2.key;

L027 | Unqualified reference 't1' found in select with more
                       | than one referenced table/view.

[tunetheweb]

I can't repeat that error, and get no linting errors with that SQL on this branch.

[R7L208]

I get it on all queries where the hint function takes a table as a parameter.

sqlfluff lint --dialect spark3 test/fixtures/dialects/spark3/select_hints.sql

==== summary ====
files:             1 violations:       11
clean files:       0 unclean files:     1
avg per file:  11.00 unclean rate:   100%
status:         FAIL
All Finished 📜 🎉!
> sqlfluff lint --dialect spark3 test/fixtures/dialects/spark3/select_hints.sql
== [test/fixtures/dialects/spark3/select_hints.sql] FAIL
L:  57 | P:  22 | L027 | Unqualified reference 't1' found in select with more
                       | than one referenced table/view.
L:  63 | P:  27 | L027 | Unqualified reference 't1' found in select with more
                       | than one referenced table/view.
L:  69 | P:  20 | L027 | Unqualified reference 't2' found in select with more
                       | than one referenced table/view.
L:  76 | P:  26 | L027 | Unqualified reference 't1' found in select with more
                       | than one referenced table/view.
L:  82 | P:  22 | L027 | Unqualified reference 't2' found in select with more
                       | than one referenced table/view.
L:  88 | P:  18 | L027 | Unqualified reference 't1' found in select with more
                       | than one referenced table/view.
L:  95 | P:  25 | L027 | Unqualified reference 't1' found in select with more
                       | than one referenced table/view.
L: 102 | P:  33 | L027 | Unqualified reference 't1' found in select with more
                       | than one referenced table/view.
L: 108 | P:  22 | L027 | Unqualified reference 't1' found in select with more
                       | than one referenced table/view.
L: 108 | P:  33 | L027 | Unqualified reference 't1' found in select with more
                       | than one referenced table/view.
L: 108 | P:  37 | L027 | Unqualified reference 't2' found in select with more
                       | than one referenced table/view.
All Finished 📜 🎉!

[tunetheweb]

Change looks good now. Can I just understand what the motivation for this is?

Why not just leave it as comments, since it currently doesn't "do" anything more?

Or is the plan to build this out more eventually to add syntax checking to these hints?

[R7L208]

The plan is to build it out for Syntax checking and the hint has to come after the SELECT whereas comment can go anywhere. I've left a TODO in the code but the order in which multiple hints are specified matters, particularly for join hints.