Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What is the recommended way to export huge data (1Lac plus rows). #33

Closed
gopalvirat opened this issue Sep 14, 2018 · 3 comments

Comments

@gopalvirat
Copy link

@gopalvirat gopalvirat commented Sep 14, 2018

Problem:

What is the recommended way to export huge data (1 Lac plus rows).
As the data will be processing line by line, what is the best possible way to handle this.

shopify_data_export_screengrab

@gopalvirat gopalvirat changed the title What the recommended way to export huge data (1Lac plus rows). What is the recommended way to export huge data (1Lac plus rows). Sep 14, 2018
@JordanVerasamy

This comment has been minimized.

Copy link
Contributor

@JordanVerasamy JordanVerasamy commented Sep 14, 2018

There's really no way to avoid making 1 API call per customer -- the Magento SOAP API does not support bulk retrieval of customer info. You have to get customer metadata in bulk and then make 1 call per customer to get the detailed data.

Are you getting timeouts? Or is it just slow and you wish it was faster?

@gopalvirat

This comment has been minimized.

Copy link
Author

@gopalvirat gopalvirat commented Sep 17, 2018

@JordanVerasamy currently getting no timeouts. It is just slow and I want it to be a little faster.

@surajreddy

This comment has been minimized.

Copy link
Contributor

@surajreddy surajreddy commented Oct 3, 2018

@gopalvirat I've created a feature request based on this to retrieve data in parallel: #66

@surajreddy surajreddy closed this Oct 3, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.