Shared files for Campaigns

[dww100]

We should have a mechanism of specifying resources (files) needed by all runs in a Campaign (I'm thinking of calling them 'fixtures'). Example:
Running MD simulations with the same initial structure and topology but different cutoffs.

[raar1]

OK. This would presumably be needed by whatever middleware e.g. FabSim) so that it knows what files need to be transferred to the compute resources? In fact, before we even get to the stage of identifying shared files we already need a means of indicating what files get sent to the runs directory.

The current test apps do not have any external file dependencies - they only need the "input file" (created by the encoder) which is already in the run_dir. We may need to beef up e.g. cannonsim so that its input file refers to some other external parameters file. Then perhaps we add a "Needs" list to Campaign, somewhere, that contains the list of all files that will need to be copied into the run directories. If there is overlap, it would not need to copy the same file a hundred times but instead would put it into a shared resources directory.

My fear is that determining what each run needs becomes another application specific point (like another encoder). Let's say that we do a BasicUQP sweep with forcefield set to one of a list of three strings. How would Campaign know that for those runs with one of those forcefields set, an extra file would need to be present in the run dir? (let's say it's a particularly obnoxious ff or something).

The (ugly) solution at present would be that when we specify the distribution of a parameter as being one of a list of options (say strings) then we also need to have an (optional) "Needs:" key added for options whose choice requires the presence of certain files.

Alternatively, we put it in a separate file, and this part is handled by the Encoder. But I don't like extra files... Especially not application specific stuff.

[dww100]

I definitely think it needs to be handled by the encoder.

I increasingly think we may need to think in terms of groups of Campaigns - so that in your case we have a campaign for each ff which specified different dependencies. These are then integrated using some mechanism when a UQP is using them for a particular analysis.

There are definitely a lot of things we need to think about with this.

I'm hoping Alex's example might prove a better development test that cannonsim. That way we don't waste time developing something that won't ever get used in anger. We should possibly look for another ready made example though.

[raar1]

We can always make a War class that consists of multiple Campaigns...

[dww100]

:)

I spent about an hour this morning trying to think of non aggressive sounding higher level term. Best I came up with was Plan.

[raar1]

It might be possible to make Campaigns contain other Campaigns in some way if we're careful about how we do it. Could lead to unmitigated horror though.

[dww100]

😵