Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rework the Processing into a DAG #19

Open
dvfeinblum opened this issue Feb 23, 2019 · 0 comments
Open

Rework the Processing into a DAG #19

dvfeinblum opened this issue Feb 23, 2019 · 0 comments
Assignees
@dvfeinblum
Labels
enhancement New feature or request
Projects
Bernoulli

Comments

@dvfeinblum
Copy link
Owner

dvfeinblum commented Feb 23, 2019 •
edited
Loading

Is your feature request related to a problem? Please describe.
Async is cool, but multiprocessing is better. Luigi Airflow is dope so let's implement it.

Describe the solution you'd like
Gut all of the async stuff (should be as easy as reverting to pre-async code), install airflow, and create some tasks.

Describe alternatives you've considered
I could use this as an opportunity to learn Airflow, but I'd rather improve my luiginess. I changed my mind.

Additional context
One of the reasons I changed my mind is because Airflow has more features than Luigi. And also we're porting our pipeline over to Airflow at work so this is like, studying or something.
@dvfeinblum dvfeinblum added this to To do in Bernoulli Feb 23, 2019
@dvfeinblum dvfeinblum self-assigned this Feb 23, 2019
@dvfeinblum dvfeinblum added the enhancement New feature or request label Feb 23, 2019
@dvfeinblum dvfeinblum moved this from To do to In progress in Bernoulli Jul 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dvfeinblum dvfeinblum

Labels
enhancement New feature or request
Projects
Bernoulli
  
In progress
Milestone
No milestone
Development

No branches or pull requests
1 participant
@dvfeinblum