Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to Export to csv file? #30

Closed
stevenluoxiaohua opened this issue Jul 31, 2016 · 3 comments
Closed

How to Export to csv file? #30

stevenluoxiaohua opened this issue Jul 31, 2016 · 3 comments

Comments

@stevenluoxiaohua
Copy link

`p.stream('select * from users')

.pipe(etl.stringify(0,null,true))
.pipe(etl.toFile('txt/test.txt'))
.promise()
.then(function() {
console.log('done')
})
.catch(function(e) {
console.log('error',e);
});`

I'm try like this, but how to do next?

@ZJONSSON
Copy link
Owner

A very simple example would be something like:

// assuming mysql client for the select
client.query('select * from users').stream()
  .pipe(etl.map(function(d) {
    // if we haven't sent headers, we start by doing so
    if (!this.headers)
      this.headers = this.push('"'+Object.keys(d).join('","')+'"\n') || true;
    return '"'+Object.keys(d).map(key => d[key]).join('","')+'"\n';
  })
  .pipe(etl.toFile('txt/text.txt'))
  .promise()
  .then(...)

However this does not properly escape the fields. A safer bet would be to use something like csv-write-stream which I believe you can use like this:

var csvWriter = require('csv-write-stream');

client.query('select * from users').stream()
  .pipe(csvWriter())
  .pipe(etl.toFile('txt/text.txt'))
  .promise()
  .then(...)

Let me know if this works or not for you

@stevenluoxiaohua
Copy link
Author

thanks,
It works ok, code as follows

`var pool = mysql.createPool({
host: 'localhost',
port: '3306',

database: 'gss'

});

var p = etl.mysql.execute(pool);

p.stream('select * from test')
.pipe(csvWriter({
separator: ',',
newline: '\n',
headers: undefined,
sendHeaders: false
}))
.pipe(etl.toFile('txt/text.txt'))
.promise()
.then(function () {
console.log('done')
});`

now, I try insert to database using mysql

@ZJONSSON
Copy link
Owner

ZJONSSON commented Aug 1, 2016

see #31 for the other question

@ZJONSSON ZJONSSON closed this as completed Aug 1, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants