Skip to content

Wip scope populator#3965

Draft
jamesstarr wants to merge 9 commits intoapache:mainfrom
jamesstarr:wip_scope_populator
Draft

Wip scope populator#3965
jamesstarr wants to merge 9 commits intoapache:mainfrom
jamesstarr:wip_scope_populator

Conversation

@jamesstarr
Copy link
Contributor

No description provided.

Adding SqlQueryScopes to encapsulate the data for resolving
SqlNode to SqlValidatorScope and SqlValidatorNamespaces.  Moving
the api for resolving scopes from SqlValidator to SqlQueryScopes.
… using

a generic Fuction in SqlValiator.Config.
Adding NamespaceBuilder to formalize the api for customizing/reusing
namespaces downstream.
@julianhyde
Copy link
Contributor

Looks pretty good.

I'm tempted to make the following next steps:

  • Rework it to make SqlValidator implement ScopeMap, and reduce the amount of noise. (I take your point that maybe the interfaces should be decoupled eventually.)
  • Make ScopeMapImpl package-private
  • Make ScopeMapImpl it immutable (with the mutable bits in a builder)
  • Address the Checkerframework issues (yes they are painful, but they generally point to real problems in lifecycle)

…is allows

for overriding validateFeature.

Removing unused getter from ValidatorAggStuff.
@jamesstarr
Copy link
Contributor Author

jamesstarr commented Sep 17, 2024

Looks pretty good.

I'm tempted to make the following next steps:

  • Rework it to make SqlValidator implement ScopeMap, and reduce the amount of noise. (I take your point that maybe the interfaces should be decoupled eventually.)
  • Make ScopeMapImpl package-private
  • Make ScopeMapImpl it immutable (with the mutable bits in a builder)
  • Address the Checkerframework issues (yes they are painful, but they generally point to real problems in lifecycle)

@julianhyde, the ScopeMap interface is already read-only, which provides most of the clarity we gain from immutability. The additional clarity from introducing an explicit immutable version of ScopeMap is marginal, while the complexity introduced is significant. This is due to your desire for a redelegation layer in SqlValidator, the existing circular dependencies between ScopeMap, namespaces, and SqlValidator, and the non-trivial methods in ScopeMapImpl, which need to support downstream overriding while not have code duplication. In addition, the scopeMap reference would have to mutate within SqlValidatorImpl, which would create a complex lifecycle and essentially negate the intended clarity.

Allowing downstream projects to override ScopeMapImpl is why it's currently package-scoped. Moreover, several methods in ScopeMap and ScopeMapImpl are public or protected, so we'd need to provide an API for overriding those methods moving forward, unless we plan to offer an upgrade path for projects that have customized them.

Given the complexity of these interactions and the time it would take to implement, I’m not willing to introduce an immutable version of ScopeMap at this time. However, the work on the ScopeMap interface will make it easier to add an immutable version later, once the circular dependencies are resolved.

I can add a redegalation layer to SqlValidator, but please be very explicit on why you want this, because I do not understand. All it seems to do is adding 200 lines of boilerplate while making api opaque and creating future work.

@sonarqubecloud
Copy link

@jamesstarr
Copy link
Contributor Author

@julianhyde Any thoughts on wether this is worth going forward?

@jamesstarr
Copy link
Contributor Author

@julianhyde Would it be ok if asked @mihaibudiu to review this?

@mihaibudiu
Copy link
Contributor

I can try to review it, but @julianhyde knows at least 10^6 times more about this software than I do

@jamesstarr
Copy link
Contributor Author

@mihaibudiu Lets wait for @julianhyde blessing to go forward before. I do not want to waste your time.

@mihaibudiu
Copy link
Contributor

Reading code is a great way of learning about how the system works, and since I only have a vague understanding of the validator, it could be useful to me. The main problem is that I couldn't come up with most arguments that @julianhyde made, because I don't understand many of the reasons behind the current design and their implications.

@github-actions
Copy link

github-actions bot commented Dec 5, 2024

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 90 days if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@calcite.apache.org list. Thank you for your contributions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants