Problems trying to use SGE with MappingSampler

[Quim98]

Hello,
I've been trying to upgrade my pyABC scripts from a parallel to a distributed execution of simulations and I've failed to do so for the moment. Since I will execute my script in a cluster that uses an UGE Batch System I thought the best approach would be to use pyabc.sge.SGE() with pyabc.sampler.MappingSampler(). In my code I have the following line:

abc = pyabc.ABCSMC(model, prior, cost_ABC, sampler=pyabc.sampler.MappingSampler(map_=pyabc.sge.SGE(tmp_directory="data/ABC_SMC/tmp", memory='5G', time_h=1, name='pyabc_inf_sim', queue='short-centos79,long-centos79', num_threads=1), mapper_pickles=True), population_size=pyabc.populationstrategy.AdaptivePopulationSize(start_nr_particles=pop_size_init, mean_cv=E_cv, max_population_size=pop_size_max))

Which obviously give the error 'SGE' object is not callable. Reading the documentation it seems i should use the .map(function, array) method after calling pyabc.sge.SGE but then I don't know which array should I input? Shouldn't this array be the different parameter inputs found in a population at a certain time step, so it's not an input I should specify now, no? Am I missing something else? Maybe it would be nice to have an example on how to implement pyabc.sge.SGE() with pyabc.sampler.MappingSampler(). For the rest of things I used pyABC I found the documentation to be very informative though so kudos to all the developers!! 💯

Thank you very much in advance,
Quim

[stephanmg]

Hi, could you post your ~/.parallel configuration file?

[stephanmg]

Ping @Quim98