Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ESQL QL Grand Refactor #106679

Open
11 tasks
nik9000 opened this issue Mar 22, 2024 · 2 comments
Open
11 tasks

ESQL QL Grand Refactor #106679

nik9000 opened this issue Mar 22, 2024 · 2 comments
Assignees
Labels
:Analytics/ES|QL AKA ESQL >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)

Comments

@nik9000
Copy link
Member

nik9000 commented Mar 22, 2024

Description

When we built ESQL we built it on top of QL's core, which itself was refactored fairly quickly out of SQL. It's what we had time for and it had a lot of goodies. But it brought a lot of baggage. SQL and EQL live on top of _search and ESQL doesn't, for example.

  • Make a copy of the ql project and make esql depend only on this.
  • Find all of the tests in SQL, EQL, and QL that we implicitly rely on from ESQL and port them to ESQL (at least optimizer rules tests + expression translator tests)
  • refactor/trim the Expression tree
  • Unify data types management (ie. avoid mix ofDataTypes and EsqlDataTypes)
  • Unify type resolution management (ie. avoid mix of TypeResolutions and EsqlTypeResolutions)
  • Replace EvalMapper with calls to EvaluatorMapper#toEvaluator
  • refactor/trim the ExpressionTranslators
  • Move everything from EvaluatorMapper to ESQL's Expression superclass
  • Double-check exception management (ie. make sure that all the exceptions thrown are EsqlClientException, and not only QlClientException)
  • Double-check the usage of QL optimization rules (see QL OptimizerRules vs ESQL OptimizerRules)
  • Have our own LogicalPlans and PhysicalPlans
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Mar 22, 2024
@nik9000
Copy link
Member Author

nik9000 commented Mar 25, 2024

  • Double-check exception management (ie. make sure that all the exceptions thrown are EsqlClientException, and not only QlClientException)

The rest of Elasticsearch doesn't really use special exceptions for a project. I think it'd be worth seeing if we should standardize on the assortment of exceptions the rest of Elasticsearch uses. Not sure we'd actually do it, but worth checking.

elasticsearchmachine pushed a commit that referenced this issue May 16, 2024
…108609)

Part of #106679

Add tests for our query translators for binary comparisons and ranges.

Turns out, our query translators, which we use whenever we push down
filters to Lucene, have only partial test coverage in some of the
LocalPhysicalOptimzerTests. We need to expand test coverage, so this
sets up the necessary plumbing and adds the first query translator spec
tests.
nicktindall pushed a commit to nicktindall/elasticsearch that referenced this issue May 17, 2024
…lastic#108609)

Part of elastic#106679

Add tests for our query translators for binary comparisons and ranges.

Turns out, our query translators, which we use whenever we push down
filters to Lucene, have only partial test coverage in some of the
LocalPhysicalOptimzerTests. We need to expand test coverage, so this
sets up the necessary plumbing and adds the first query translator spec
tests.
parkertimmins pushed a commit to parkertimmins/elasticsearch that referenced this issue May 17, 2024
…lastic#108609)

Part of elastic#106679

Add tests for our query translators for binary comparisons and ranges.

Turns out, our query translators, which we use whenever we push down
filters to Lucene, have only partial test coverage in some of the
LocalPhysicalOptimzerTests. We need to expand test coverage, so this
sets up the necessary plumbing and adds the first query translator spec
tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)
Projects
None yet
Development

No branches or pull requests

5 participants