'Non-aggregate SELECT expression(s) not part of GROUP BY' Error When Using LATEST_BY_OFFSET #5114
Comments
|
Tried in ksqdb 0.8.1 and, whilst the query successfully runs, the table is not populated and these errors appear in the logs: |
|
are you saying that in 0.7.0 the query is rejected at compile time? That is because |
|
I can't say for sure, but I think this will probably fix the error, yes. Sorry, I hadn't realised that the function didn't exist in 0.7.0. However, the error given doesn't state that the function doesn't exist, only that it should be in the
|
|
I'm running into the same error message when trying to use |
fixes: confluentinc#5114 Added test to ensure the latest version of ksqlDb gives a meaningful error message if an unknown function is called with a non-aggregate select expression. Previous versions failed with somewhat confusing error e.g. ``` Non-aggregate SELECT expression(s) not part of GROUP BY: [WONT_FIND_ME(ID)] ``` Now fails with: ``` Can't find any functions with the name 'WONT_FIND_ME' ```
5.4.1 does not include |
|
Hey all. The issues here seem to be only pertaining to versions of KSQL that don't have Hence I've raised a PR to ensure a more helpful error message is returned if the method doesn't exist. Merging that PR will close this issue. Please feel free to raise another issue if anyone feels there specific issue has not been addressed. |
|
Hi @big-andy-coates, thanks for clarifying that this function doesn't work with tables; the documentation I read wasn't clear on this point. |
|
@big-andy-coates @brightneuron I think my larger problem is To be clear, the source I would be creating the table from is a stream. |
|
It can used in a Notice the The reason the function doesn't work on table sources is because table can have data retracted from them, requiring the derived table to undo some previous update. This is not feasible for many aggregate functions, including LATEST_BY_OFFSET, as they would need to maintain a full history of all values ever seen. |
What does your source data look like? If each record contains the values you want then you may be able to just import it as a change log topic, i.e. via a CREATE TABLE statement, rather than a CREATE STREAM. If the topic is not correctly keyed, you may need to first import as a stream, repartition and reimport the sink topic as a table. |
|
Hi, I'm getting the same error using CREATE TABLE with the source being a stream |
…5271) * test: add group by test to ensure useful error on unknown function fixes: #5114 Added test to ensure the latest version of ksqlDb gives a meaningful error message if an unknown function is called with a non-aggregate select expression. Previous versions failed with somewhat confusing error e.g. ``` Non-aggregate SELECT expression(s) not part of GROUP BY: [WONT_FIND_ME(ID)] ``` Now fails with: ``` Can't find any functions with the name 'WONT_FIND_ME' ``` Co-authored-by: Andy Coates <big-andy-coates@users.noreply.github.com>
|
Example below, should this be giving an error (using 0.8.1)? CREATE TABLE 600_customer_events_agg WITH (PARTITIONS=6) AS SELECT |
|
Hi Andy, my table is based on a stream of incoming flight position events ("incoming_flight_events"). Each event in this stream contains partial flight position data only, represented in columns such as longitude, latitude, ground speed, altitude, etc). So my hope was to create a table that could summarise each detected flight (contact) with a combination of some aggregate data (e.g. no. of associated events) along with the last known location data. My table statement looks something like this (example error-inducing statements "commented out"): Unfortunately if I include the LATEST_BY_OFFSET parameters, the statement fails as above. This seems counter-intuitive, but I'm a relative ksqldb newbie. |
|
Hi @big-andy-coates, shouldn't this issue be reopened? |
|
Never mind. FYI for others with the same confusion - I clarified via Community Slack that this UDAF is available on KSQL standalone 0.8.1, however this version isn't yet packaged into Confluent Platform 5.5.0. And unfortunately KSQL throws that unhelpful error message rather than advising the UDAF doesn't exist. |
|
Hit the same problem. The documentation should clearly state in which version of KSQLDB each function was introduced. And it is NOT clear what version of KSQLDB is packed in Confluent Platform 5.5, either the docker images should be versioned according to source version, or there should be a clear way to retrieve version from the Confluent Control Center. just to find out that LATEST_BY_OFFSET is not included in Confluent Platform 5.5. PLEASE, invest more time into documentation, TODOs in the docs are looking ridiculous. |
|
LATEST_BY_OFFSET aggregate function works well in Confluent Plataform 5.5.1-2.12. |
|
I'm getting this error in Confluent Cloud years later, where I know LATEST_BY_OFFSET is supported. |
Describe the bug
I wanted to create a table which joined several rows into one collecting a single column into an array by using
COLLECT_LIST. I also wanted that table to be keyed by a single column, but still have the other columns visible. So using a largeGROUP BYon all the columns wasn't an option.I figured the aggregate function
LATEST_BY_OFFSETwas the closest to what I wanted to do as it would return the latest column, even though they would always be the same. However, running the following statement:And got the following error:
Ideally I'd be able to group by one column and use non-aggregated columns in the same query but I thought using an aggregated column in its place would work here, but it appears
LATEST_BY_OFFSETisn't considered to produce an aggregated result in this case.Using
TOPKdoes work, however:To Reproduce
Steps to reproduce the behavior, include:
Sample queries above.
Sample data (note multiple rows can exist for a single album representing each artist):
Expected behavior
I expect the table to be created without error.
Actual behaviour
I get the following error:
The text was updated successfully, but these errors were encountered: