Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copy table to parquet does not work in Node.js environment #1977

Closed
sashakorman opened this issue Jul 9, 2021 · 5 comments
Closed

Copy table to parquet does not work in Node.js environment #1977

sashakorman opened this issue Jul 9, 2021 · 5 comments

Comments

@sashakorman
Copy link

What does happen?
Trying to export a table to parquet file using COPY TO syntax in Node.js environment.
Got the following error:

[Error: Catalog Error: Copy Function with name parquet does not exist!
Did you mean "csv"?] {
errno: -1,
code: 'DUCKDB_NODEJS_ERROR'
}

What should happen?
Table should be exported into parquet file

To Reproduce
Run the following code.

const duckdb = require('duckdb');

const db = new duckdb.Database(':memory:'); 
const connection = db.connect();
connection.run(`CREATE TABLE posix (path VARCHAR, size BIGINT)`);
connection.run(`INSERT INTO posix (path, size) VALUES ('/work', 1)`);
connection.run(`COPY posix TO 'out.csv' (FORMAT 'CSV')`);
connection.run(`COPY posix TO 'out.parquet' (FORMAT 'PARQUET')`);
connection.all(`COPY posix TO 'out.parquet' (FORMAT 'PARQUET')`, (err, res) => {
    if (err) {
      throw err;
    }
    console.log(res);
  });
  • Export to CSV does work.

  • Export to PARQUET does work using CLI.

Environment:

  • OS: Amazon Linux 2
  • DuckDB version: 0.2.7
  • Node version: 16.4.2
@hannes
Copy link
Member

hannes commented Jul 12, 2021

Yes, we are currently not shipping the Parquet extension with the node client. We probably should do that.

@hamilton
Copy link
Contributor

hamilton commented Dec 19, 2021

Found this issue after searching. Could someone point us to where in the repo we might make this happen? I would be happy to submit a patch that does this to get the ball rolling if it is straightforward.

@hannes
Copy link
Member

hannes commented Dec 19, 2021

Getting a PR ready hang on

@davidtheclark
Copy link

I'm using Node.js SDK v0.3.3, and I'm successfully copying into Parquet files. Should this issue be closed?

@hannes
Copy link
Member

hannes commented Sep 19, 2022

Yes, we also ship the Parquet reader as a loadable extension now

@hannes hannes closed this as completed Sep 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants