[PLAT-435] allow dict rewrites in ch without create temp table privileges, add docs

[pjain1]

• Add EscapeQualifiedIdentifier to escape dot-separated identifiers (e.g. db.table) which may be provided for lookup_table property.
• Change LookupSelectExpr to query the dictionary table directly instead of using dictionary() table function, which requires CREATE TEMPORARY TABLE privilege.
• Docs for lookup feature.

Checklist:

• Covered by tests
• Ran it and it works as intended
• Reviewed the diff before requesting a review
• Checked for unhandled edge cases
• Linked the issues it closes
• Checked if the docs need to be updated. If so, create a separate Linear DOCS issue
• Intend to cherry-pick into the release branch
• I'm proud of this work!

[pjain1]

now this directly reads from the dictionary table, may be we should make this configurable if the dictionary table as dictionary() may directly read from memory if the dict is loaded in memory so a minor optimization probably.

[begelundmuller]

I'm not sure I understand this issue, why can't we keep dictionary(%s)? And how would you propose to make it configurable?

[pjain1]

using dictionary() is the main issue for which PR is raised, its a table function and on the table functions page it says You can't use table functions if the [allow_ddl](https://clickhouse.com/docs/operations/settings/settings#allow_ddl) setting is disabled. Customer queries are failing with

Not enough privileges. To execute this query, it's necessary to have the grant CREATE TEMPORARY TABLE ON *.*. (ACCESS_DENIED)

Thus directly reading from the underlying dictionary table. By making it configurable I meant using an instance config flag using which we will choose if to use table function or directly read from dict table.

[begelundmuller]

Would be nice to avoid a config flag for such an edge case if possible. But it's okay if it's needed.

Does dictionary() impact performance a lot? If it does, then maybe we should just tell users that Clickhouse requires DDL access to use dictionaries in Rill? If it doesn't, then I think this is fine and we can just not use the table func.

[pjain1]

Don't have empirical evidence around performance as in practice dictionary tables are really small so could not see much difference. I think its fine to keep this not configurable.

[pjain1]

Instead of automatically parsing dots in lookup_table, should we add an optional lookup_database property since clickhouse table names can have dots in them. Since its very rare and may also require quoting full table name while doing dictGet, I just kept it simple.

[begelundmuller]

I think it's fine to keep it simple – see my other comment

[begelundmuller]

I think it's fine to keep it simple – see my other comment

[begelundmuller]

I'm not sure I understand this issue, why can't we keep dictionary(%s)? And how would you propose to make it configurable?