New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output file matches when defining a task #3

Open
ODiogoSilva opened this Issue Jul 12, 2017 · 1 comment

Comments

2 participants
@ODiogoSilva
Contributor

ODiogoSilva commented Jul 12, 2017

I noticed that when defining a task:

// this is a kiss example of how tasks work with shell
const simpleTask = task({
  output: '*.txt', // checks if output file matches the specified pattern
  params: 'test_file.txt',  //defines parameters to be passed to the
    // task function
  name: 'This is the task name' //defines the name of the task
}, function(resolvedProps) {
    const params = resolvedProps.params
    return 'touch ' + params
  }
)

It seems that only one output file can be matched for a given extension?

output: '*.txt', // checks if output file matches the specified pattern

Is there a way to match multiple files of the same extension?

@tiagofilipe12 tiagofilipe12 self-assigned this Jul 12, 2017

@tiagofilipe12

This comment has been minimized.

Member

tiagofilipe12 commented Jul 12, 2017

In its current state watermill crawls the DAG (or simply tree of tasks) and searches for the first match for that pattern. So if you have something like:

Task1 output --> *.txt
Task2 output --> *.txt
Task3 input --> *.txt

and your pipeline is something like join(Task1, Task2, Task3)

Task3 input will match Task2 output *.txt.

Although it can take multiple output patterns such as defined here:

output: ['1.txt', '2.txt'].map(suffix => `*.${suffix}`),

Same way it can take several inputs:

input: ['1.txt', '2.txt'].map(suffix => `*.${suffix}`),

This is something worth mentioning here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment