Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Primitive Keys: Remove WITH(KEY='z') #3537

Closed
big-andy-coates opened this issue Oct 11, 2019 · 1 comment · Fixed by #5363
Closed

Primitive Keys: Remove WITH(KEY='z') #3537

big-andy-coates opened this issue Oct 11, 2019 · 1 comment · Fixed by #5363

Comments

@big-andy-coates
Copy link
Contributor

big-andy-coates commented Oct 11, 2019

With #3536 complete we can deprecate the use of WITH(KEY='x') in all statements, as the user can now use whatever name they like for their key column.

Internally, we can still choose to maintain a track of fields in the value that are copies of the key. However, this becomes an internal optimization. However, I think the ROI on this is very low. We already know there are a lot of bugs in the key field handling code and it adds a LOT of complexity to KSQL. Plus, to support this properly we'd need to extend the key field functionality to be able to track multiple copies of the key in the value, which is more work and more complexity and more space for bugs.

The way I see it is that the WITH(KEY) syntax was introduced to allow users to use a nicely named column in their SQL statements, e.g.

CREATE TABLE FOO (ID INT, NAME STRING) WITH(KEY='ID', ...);
SELECT COUNT() FROM FOO GROUP BY ID;

In the above select the user can use ID in the GROUP BY, which is more descriptive that GROUP BY ROWKEY, (which is equivalent).

However, with the ability to name the key column you can now do:

CREATE TABLE FOO (ID INT KEY, NAME STRING) WITH(...);
SELECT COUNT() FROM FOO GROUP BY ID;

Which has the same effect, without the need to duplicate the key into the value.

@big-andy-coates
Copy link
Contributor Author

As part of this work we should update partition-data.rst

@big-andy-coates big-andy-coates added this to To do in Primitive Keys via automation Oct 25, 2019
@big-andy-coates big-andy-coates added this to To do in Language changes via automation Dec 16, 2019
@big-andy-coates big-andy-coates moved this from Backlog to To do in Primitive Keys Mar 25, 2020
@big-andy-coates big-andy-coates added this to the 0.10.0 milestone Apr 24, 2020
@big-andy-coates big-andy-coates moved this from To do to In progress in Primitive Keys May 14, 2020
@big-andy-coates big-andy-coates changed the title Primitive Keys: Deprecate WITH(KEY='z') Primitive Keys: Remove WITH(KEY='z') May 14, 2020
big-andy-coates added a commit to big-andy-coates/ksql that referenced this issue May 14, 2020
fixes: confluentinc/ksql#3537

implements: See [KLIP-25](https://github.com/confluentinc/ksql/blob/master/design-proposals/klip-25-removal-of-with-key-syntax.md)

This change removes the `WITH(KEY)` syntax which previously allowed users to specify a value column that could act as an alias for the key column. This allowed a more user friendly name to be used for the key column, at the expense of requiring a copy of the key data in the value.

With the new 'any key name' feature, the key columns themselves can be given appropriate names, removing the need for this aliasing functionality.  See [KLIP-25](https://github.com/confluentinc/ksql/blob/master/design-proposals/klip-25-removal-of-with-key-syntax.md) for more details.
@big-andy-coates big-andy-coates moved this from In progress to Review in progress in Primitive Keys May 15, 2020
Primitive Keys automation moved this from Review in progress to Done May 18, 2020
Language changes automation moved this from To do to Done May 18, 2020
big-andy-coates added a commit that referenced this issue May 18, 2020
* feat: drop WITH(KEY) syntax

fixes: #3537

implements: See [KLIP-25](https://github.com/confluentinc/ksql/blob/master/design-proposals/klip-25-removal-of-with-key-syntax.md)

This change removes the `WITH(KEY)` syntax which previously allowed users to specify a value column that could act as an alias for the key column. This allowed a more user friendly name to be used for the key column, at the expense of requiring a copy of the key data in the value.

With the new 'any key name' feature, the key columns themselves can be given appropriate names, removing the need for this aliasing functionality.  See [KLIP-25](https://github.com/confluentinc/ksql/blob/master/design-proposals/klip-25-removal-of-with-key-syntax.md) for more details.

Co-authored-by: Andy Coates <big-andy-coates@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Development

Successfully merging a pull request may close this issue.

1 participant