Stream files directly to Snowflake

[kri5t]

Please answer these questions before submitting your issue. Thanks!

1. What version of NodeJS are you using (node --version and npm --version)?
   16.4.2

2. What operating system and processor architecture are you using?
   Docker (Alpine-node)

3. What are the component versions in the environment (npm list)?
   Snowflake-sdk: 1.6.3

It seems to me that you have hard-coded this Snowflake connector to only work with the filesystem when sending files to Snowflake.

You are opening a fileStream from the s3_util, azure_util and google_util. I would rather upload using my own stream (not from the file system).

It doesn't seem like a very flexible way of sending data to Snowflake. Instead it would be great if you allowed the user to attach their own stream (like Google BigQuery does it). This way we don't have to bloat our docker image file system to upload a file to Snowflake.

As I see it the only way I can do this right now is picking up the stageInfo details using this library and then extracting that info and uploading the stream myself. Is there any way for me to issue a request to get the internal stage info?

I see that the request being issued in the client is: PUT file:///filesystem/002.avro @TESTDB.PUBLIC.NEW_STAGE
And I am guessing this is what will allow me to fetch the actual details.

I can see that info when I stop the debugger here on stageInfo.

[kri5t]

It also seems like this is the way your Java SDK is working and I think it brings so much more freedom for the developers:
https://docs.snowflake.com/en/user-guide/jdbc-api.html#interface-snowflakeconnection

Is it something you are planning to bring to node.js?

[ak--47]

was this ever actually completed?

[sfc-gh-dszmolka]

no, and i was not aware of this closed issue - thank you for drawing attention.
this enhancement request is now tracked under #834 for both PUT and GET - will keep 834 updated with any new information should they become available