feat: Support SQL SELECT * ILIKE wildcard syntax

[alexander-beedie]

Following on from #17109, which added RENAME and REPLACE modifiers:

New

• Added support for the SELECT * ILIKE … wildcard select-modifier¹.
  This is a pattern-match on column names, using SQL "%" and "_" glob chars.

Also

• Fixed interaction of ORDER BY and REPLACE modifier (should sort on post-replace values).
• Fixed use of ordinal values with ORDER BY clause if also using SELECT * EXCLUDE.
• Improved use of ORDER BY on columns targeted by RENAME modifier.
• Some code refactoring/consolidation to improve clarity around select-modifiers.
• Extended select-modifier unit tests to cover additional permutations/combinations of modifiers.

Note

With this final addition, we will support all SELECT * wildcard modifiers for the 1.0 🎉

SELECT * EXCLUDE …
SELECT * ILIKE …
SELECT * RENAME …
SELECT * REPLACE …

Examples

import polars as pl

df = pl.DataFrame({
    "ID": [333, 666, 999],
    "FirstName": ["Bruce", "Diana", "Clark"],
    "LastName": ["Wayne", "Prince", "Kent"],
    "Address": ["The Batcave", "Paradise Island", "Fortress of Solitude"],
    "City": ["Gotham", "Themyscira", "Metropolis"],
})

Demonstrate use of SELECT * ILIKE:

df.sql("""
  SELECT * ILIKE '%a%e%'
  FROM self
  ORDER BY FirstName
""")
# shape: (3, 3)
# ┌───────────┬──────────┬──────────────────────┐
# │ FirstName ┆ LastName ┆ Address              │
# │ ---       ┆ ---      ┆ ---                  │
# │ str       ┆ str      ┆ str                  │
# ╞═══════════╪══════════╪══════════════════════╡
# │ Bruce     ┆ Wayne    ┆ The Batcave          │
# │ Clark     ┆ Kent     ┆ Fortress of Solitude │
# │ Diana     ┆ Prince   ┆ Paradise Island      │
# └───────────┴──────────┴──────────────────────┘

Combine ILIKE with RENAME, sort with ordinal-based ORDER BY:

df.sql("""
  SELECT * ILIKE '%I%' RENAME (FirstName AS Name) 
  FROM self
  ORDER BY 3 DESC
""")
# shape: (3, 3)
# ┌─────┬───────┬────────────┐
# │ ID  ┆ Name  ┆ City       │
# │ --- ┆ ---   ┆ ---        │
# │ i64 ┆ str   ┆ str        │
# ╞═════╪═══════╪════════════╡
# │ 666 ┆ Diana ┆ Themyscira │
# │ 999 ┆ Clark ┆ Metropolis │
# │ 333 ┆ Bruce ┆ Gotham     │
# └─────┴───────┴────────────┘

Correctly apply ORDER BY on a column targeted by the RENAME modifier:

df.sql("""
  SELECT * EXCLUDE (ID, City, LastName) RENAME FirstName AS Name
  FROM self
  ORDER BY Name
""")
# shape: (3, 2)
# ┌───────┬──────────────────────┐
# │ Name  ┆ Address              │
# │ ---   ┆ ---                  │
# │ str   ┆ str                  │
# ╞═══════╪══════════════════════╡
# │ Bruce ┆ The Batcave          │
# │ Clark ┆ Fortress of Solitude │
# │ Diana ┆ Paradise Island      │
# └───────┴──────────────────────┘

Footnotes

1. Note that these SELECT * wildcard options have become very popular in the last couple of years, with DuckDB, Snowflake, Databricks, and other vendors jumping on to the syntax, but PostgreSQL (our usual syntax target) actually has yet to implement it. However, given its popularity I think it's important that we have it (and it's purely additive; we lose nothing by supporting it) ✌️ ↩

[codecov]

Codecov Report

Attention: Patch coverage is 97.75281% with 2 lines in your changes missing coverage. Please review.

Project coverage is 80.82%. Comparing base (6518a80) to head (8a77789).
Report is 7 commits behind head on main.

Files	Patch %	Lines
crates/polars-sql/src/context.rs	97.75%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #17169      +/-   ##
==========================================
- Coverage   80.94%   80.82%   -0.13%     
==========================================
  Files        1464     1464              
  Lines      191956   192043      +87     
  Branches     2742     2744       +2     
==========================================
- Hits       155385   155224     -161     
- Misses      36062    36308     +246     
- Partials      509      511       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.