Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DSQ] Passing multiple files as different "tables" and doing joins #132

Closed
TheKnarf opened this issue Dec 22, 2021 · 3 comments · Fixed by multiprocessio/dsq#2
Closed

[DSQ] Passing multiple files as different "tables" and doing joins #132

TheKnarf opened this issue Dec 22, 2021 · 3 comments · Fixed by multiprocessio/dsq#2

Comments

@TheKnarf
Copy link

TheKnarf commented Dec 22, 2021

For the DSQ tool:

Seems like this tool would be a lot more useful if I could pass multiple files, naming each of them as a "table" name, and then doing joins on the data. Would be especially useful if one could do this with different datatype, maybe I have a couple of Excel documents, a csv and some json data coming from different datasources and I'd like to run a query over all of them.

Current workflows involves writing data importers to get all of the data into a proper SQL database server, and then running queries against to generate a report.

@TheKnarf TheKnarf changed the title Passing multiple files as different "tables" and doing joins [DSQ] Passing multiple files as different "tables" and doing joins Dec 22, 2021
@eatonphil
Copy link
Member

Thanks for the suggestion! Yes this should be doable just by extending the existing filename lookup/read logic.

@eatonphil
Copy link
Member

eatonphil commented Jan 11, 2022

Hey @TheKnarf this is now available in dsq. dsq now has its own repo at https://github.com/multiprocessio/dsq. Feel free to follow the install instructions and README for multiple files there and give it a whirl.

Thanks again for your feedback!

@TheKnarf
Copy link
Author

Cool! Love that it got its own repo. Just starred the new repo. Great job!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants