-
Notifications
You must be signed in to change notification settings - Fork 93
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support imported suite modules. #1829
Comments
Things that are occurring to me as to be thought about:
Additionally, if something is changed in the imported module, how do we handle that in terms of reloads and restarts? Possible situations being:
How do we go about validating the imported suite - as "In terms of suite design, the module workflow looks to the importing suite like a single task (albeit without some of the usual runtime settings)." - so how would we capture validating graphing in the imported suite to avoid cases where it might be invalid and would break things at runtime? Do you envisage us being able to work with graphing brought in from the module in a standard way? i.e. can we do things like Do we need to do something special to avoid:
being mixed up with the result of:
|
We can do something like Python's |
If you want ultimate control of suite configuration, you should only import from version controlled location, or locations that you have full control of. However, it is sometimes advantageous to import from a moving HEAD or latest stable. In which case:
|
Which I guess means that we'll need to keep a copy of the processed out contents of the imported content in there rather than a reference to some external (possibly processed) suite files. |
We should probably support a search path + import from relative path mechanism - so that the import statement can be detached from site-specific locations. |
@arjclark (above #1829 (comment)):
I suppose we could design a solution such that restart and reload do not re-import by default, but in my view we shouldn't do that. If using a "non-revision based solution" (or similarly, extracting from HEAD of a branch in other contexts) you get what you paid for - so be aware, and be careful!
By "in terms of suite design" I mean all the suite writer has to worry about assuming that imported modules are not broken or incompatible with the importer. I suppose we could have a validation that doesn't actually do the import, but I was envisaging that validation would do a full suite parse as for a suite start-up, in which case a broken module would cause the validation to fail (c.f. importing a broken Python module).
Yes, although I haven't thought about this in much depth. As described above, the imported module does end up as a regular task family with internal dependencies, so
Yes - abort if a name collision is detected. |
@matthewrmshin (above #1829 (comment)):
Good point. We also need to take into account our plans to move the "rose suite-run" installed-suite paradigm into cylc, which separates the installed/registered location from the source location. |
|
Yes, to me at least, and kind of prompted my questions about things in relation to non-revision controlled files i.e. ones where you can't use a @MODULE-REV extension. |
(I am thinking about a system similar to Python's |
Module source locations: This should wait on migration of |
[meeting] - we agreed on the proposal as described in the updated issue description, plus previous comment on source locations. An additional thought was:
|
Closing this issue: the proposed mechanism worked reasonably well within the confines of the config file format, but we decided quite some time ago to wait on the future Python API to get proper modularity via Python itself. |
This supersedes #173.
[UPDATE] Description edited to clarify the basic concept - which is sufficient for all use cases I know of - and to remove (my) unnecessary speculation beyond that (sorry if this removes context for some comments below!)
We should support imported suite modules to allow sharing - instead of duplicating - of common (sub-)workflows.
Concept
A suite module
Proposed Interface
When this suite is parsed, the module
The text was updated successfully, but these errors were encountered: