Added validation checks for import_tasks_from

[LawrenceCheng1570]

Issue #669
Changes:
When loading import_tasks_from: file.yaml

1. If yaml.safe_load is None, replace it with an empty list
2. If yaml.safe_load returns something other than a list, raise an error saying we were expecting a list

[edublancas]

thanks for your contribution! can you add some tests? please add them here

In the first case, we wanna check that an empty file does not break initializing the DAGSpec object, in the second case, we want to test that an appropriate error is shown

let me know if you encounter issues or have questions!

[edublancas]

Hi @LawrenceCheng1570 , thanks for the contribution!

I reproduced the error. The problem is that a downstream function doesn't work with empty lists, so I think the original logic needs to change a bit

instead of this:

if imported is None:
    imported = []

it should be:

if not imported:
    path = str(self.data['meta']['import_tasks_from'])
    raise ValueError(f'expected import_tasks_from file ({path!r}) to return a list of tasks, got: {imported}')

Note that this conditional will take care of None, [], and empty dictionaries.

Please also update the test so it checks that the appropriate error message is shown: see here

the condition can be something like: assert "expected import_tasks_from" in str(excinfo.value)

[LawrenceCheng1570]

Hi @edublancas, thanks for the suggestions! It fixed my error and I just pushed my changes. Please let me know if I need to make any other edits.

[edublancas]

almost there! just minor observations

[edublancas]

looks good!

[edublancas]

as a best practice when writing tests that evaluate error messages. it's best to only include the statement that throws the error, in this case:

with pytest.raises(ValueError) as excinfo:
    DAGSpec(spec_d)

Please move everything the spec = ... statement outside the with pytest.raises line, and you can remove the spec.to_dag().render()

[LawrenceCheng1570]

Thanks @edublancas! Just pushed my changes

[edublancas]

wooo great job! thanks a lot for contributing!

[LawrenceCheng1570]

Thanks so much! Really appreciate all the help and support for my first time contributing :)