Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple inputs for a task #65

Open
1 of 2 tasks
tiagofilipe12 opened this issue Jul 12, 2017 · 1 comment
Open
1 of 2 tasks

Multiple inputs for a task #65

tiagofilipe12 opened this issue Jul 12, 2017 · 1 comment

Comments

@tiagofilipe12
Copy link
Member

tiagofilipe12 commented Jul 12, 2017

Right now a task can only resolve if one input (with the same pattern, e.g. *.txt) is given, if pattern within input renders more than one possible hit task will not properly finish. However some in order to take multiple inputs task must:

  • Control the computer resources in order not to exceed the number of available CPUs and memory. For instance, when we pass more than one input and give it to a program that uses multithreading, if many inputs are given resources may be exceeded and odd behaviors may happen.
  • There are two types of usages in which this might be useful and that render very different results:
  1. When task returns one output per input file
  2. When task returns one output for all input files given.

Note: output may in fact be several files, here I mean by output the 'results of a task'.

@tiagofilipe12
Copy link
Member Author

Part of this topic is explored in this report.

@tiagofilipe12 tiagofilipe12 self-assigned this Aug 16, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Bionode Project Board
  
In Progress
Watermill board
DAG related
Development

No branches or pull requests

1 participant