Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem with generator stament #214

Closed
tyatabe opened this issue Jan 27, 2022 · 3 comments · Fixed by #307
Closed

Problem with generator stament #214

tyatabe opened this issue Jan 27, 2022 · 3 comments · Fixed by #307
Labels
question Further information is requested

Comments

@tyatabe
Copy link

tyatabe commented Jan 27, 2022

This is a contribed example of something I'm trying to do using Snowflake sql (where this query works correctly):
SELECT 'HELLO' AS FEATURE FROM TABLE (GENERATOR(ROWCOUNT => (24))))
This is the error message I get
SQLLineageException: An Identifier is expected, got Token[value: )] instead
from the sqllineage heroku app I get this somewhat more specific message
An Identifier is expected, got Parenthesis[value: (GENERATOR(ROWCOUNT => (24)))] instead

I'm using sqllineage version 1.2.4

@reata
Copy link
Owner

reata commented Feb 19, 2022

Sorry for the late reply, been really busy recently.

GENERATOR is special from other table names. As you can see, usually table name is parsed as Identifier(https://en.wikipedia.org/wiki/Identifier_(computer_languages)). You can think of identifier as string literal.

here we have a Parenthesis, which is clearly not a string literal.

I see what generator is doing in snowflake. So what's your expected output here?

Reference for myself: https://docs.snowflake.com/en/sql-reference/functions/generator.html

@reata reata added the question Further information is requested label Feb 19, 2022
@tyatabe
Copy link
Author

tyatabe commented Apr 1, 2022

Apologies for being late, also been busy.
From this toy example what I'm expecting to get is a table with the column FEATURE containing the value HELLO 24 times. For my application I wanted to use it to generate sequences of hours between two dates, for example see this:

SET date_start_window ='2021-01-01'; SET date_end_window = '2030-12-31'; SET NUM_HOURS = (Select DATEDIFF(HOUR, $date_start_window::DATE, $date_end_window::DATE));
SELECT DATEADD( HOUR, '-' || ROW_NUMBER() OVER (ORDER BY NULL), $date_end_window::DATE) AS TS_HOUR FROM TABLE (GENERATOR(ROWCOUNT => ($NUM_HOURS))) ORDER BY TS_HOUR;

@reata
Copy link
Owner

reata commented Apr 6, 2022

Apologies for being late, also been busy. From this toy example what I'm expecting to get is a table with the column FEATURE containing the value HELLO 24 times. For my application I wanted to use it to generate sequences of hours between two dates, for example see this:

SET date_start_window ='2021-01-01'; SET date_end_window = '2030-12-31'; SET NUM_HOURS = (Select DATEDIFF(HOUR, $date_start_window::DATE, $date_end_window::DATE)); SELECT DATEADD( HOUR, '-' || ROW_NUMBER() OVER (ORDER BY NULL), $date_end_window::DATE) AS TS_HOUR FROM TABLE (GENERATOR(ROWCOUNT => ($NUM_HOURS))) ORDER BY TS_HOUR;

Yeah, thanks for explaining, I understand the GENERATOR expression and your use case now. The thing I'm more interested to know is what would you expect sqllineage to return for this syntax. A source table named 'GENERATOR', or 'GENERATOR(ROWCOUNT => ($NUM_HOURS))'? Or should we create another class for this, like we did previous for Path?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants