Support setting time range in Copy From statement

[MichaelScofield]

What type of enhancement is this?

User experience

What does the enhancement do?

As title.

Currently there's timestamp_range: Option<TimestampRange> field in the struct CopyTableRequest, but it's not set. We can set it from the SQL parser first, then check if it works.

The copy from in postgresql supports "where condition" (https://www.postgresql.org/docs/current/sql-copy.html), maybe we can do that, too. For this issue, we can extract the time range from the "where condition", and leave more general filtering in the future.

Implementation challenges

No response

[naosense]

hi @MichaelScofield ，i want to take this, would you give me some further advice?

[MichaelScofield]

@naosense Sure. The "copy from" statement is parsed in file "copy_parser.rs", you can start from there. First you might want to see how "where" can be parsed in sqlparser. If lucky, the "copy from" statement from sqlparser may already carry the "where" part. Then you need to find how to extract the desired "time range" from the filters in "where". Finally set the time range in the struct CopyTableRequest, do some test to see if it works!

[WenyXu]

hi @MichaelScofield ，i want to take this, would you give me some further advice?

I think we can add a where clause for the Copy From statement, then forward the filter of the where clause to the scan request(The Copy From statement sends a scan request to retrieve data).

[naosense]

After my initial research, I do not have much idea about solving this issue, probably because I am not familiar with the greptimedb and sqlparser. I hope someone can take over, I'm afraid this issue is beyond my ability😅.

[okJiang]

I spent quite a while trying to solve this problem. There has been a small progress, but since I am a newbie in rust, I find it difficult for me to solve rust syntax problems in a short time, such as error handling, type conversion, and syntax sugar. So I decided to post my progress here, welcome to discuss ideas.

First I successfully parsed the where clause and got the expression. As a rookie, it took me a while to get if-else to return the same type of result...

let predicate = if self.parser.parse_keyword(Keyword::WHERE) {
            Some(self
                .parser
                .parse_expr()
                .context(error::SyntaxSnafu)?;)
} else {
            None
};

Then I found that the returned expression did not meet the needs of the issue. What needs to be returned here is timestampRange. So, the next step is to convert the expression in sql-parser into timestampRange. I think what can be done is to:

1. check if the type of the expression is AnyOp or AllOp
2. check if the data type is timestamp
3. handle various >, < situations
4. ...

[MichaelScofield]

@naosense that's ok, feel free to take another good first issue!

[MichaelScofield]

@okJiang we can make things a lot simpler by restricting the allowed syntax here. The filters in copy from stmt only take one of the following 3 forms:

• where <F> and <F>
• where <F> or <F>
• where <F>

where F = <column> > | >= | < | <= | = <timestamp literal value>