Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support COPY INTO ... (SELECT fields FROM @stage) #7228

Closed
Tracked by #7444
flaneur2020 opened this issue Aug 22, 2022 · 3 comments
Closed
Tracked by #7444

Support COPY INTO ... (SELECT fields FROM @stage) #7228

flaneur2020 opened this issue Aug 22, 2022 · 3 comments
Labels
C-feature Category: feature

Comments

@flaneur2020
Copy link
Member

flaneur2020 commented Aug 22, 2022

Summary

there's a data load with transformation feature in snowflake:

/* Data load with transformation */
COPY INTO [<namespace>.]<table_name> [ ( <col_name> [ , <col_name> ... ] ) ]
     FROM ( SELECT [<alias>.]$<file_col_num>[.<element>] [ , [<alias>.]$<file_col_num>[.<element>] ... ]
            FROM { internalStage | externalStage } )
[ FILES = ( '<file_name>' [ , '<file_name>' ] [ , ... ] ) ]
[ PATTERN = '<regex_pattern>' ]
[ FILE_FORMAT = ( { FORMAT_NAME = '[<namespace>.]<file_format_name>' |
                    TYPE = { CSV | JSON | AVRO | ORC | PARQUET | XML } [ formatTypeOptions ] } ) ]
[ copyOptions ]

also, we can consider adding transform functions into the SELECT clause, so we can make some simple transforms during the COPY clause, like extract fields from JSON.

COPY INTO books FROM (SELECT parse_json($0).title, parse_json($0).url FROM @stage1)
@BohuTANG
Copy link
Member

cc @sundy-li

@BohuTANG BohuTANG added the C-feature Category: feature label Aug 22, 2022
@ClSlaid ClSlaid self-assigned this Aug 23, 2022
@ClSlaid
Copy link
Collaborator

ClSlaid commented Aug 25, 2022

Is this related to #7211? should wait for #7211 to complete first?

@BohuTANG
Copy link
Member

Let's do this simpler task first :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-feature Category: feature
Projects
Status: 📋 Backlog
Development

No branches or pull requests

3 participants