Convert Binary search to Parallel NCPU search with NEURON.

[russelljjarvis]

Hi Justas,

Where is the other binary search we were talking about in this code?

https://github.com/scrook/neuroml-db/blob/master/Import%20Scripts/model-importer/cellmodel.py#L753

The general pattern is not very hard:

Define a function to do the transformation on a list (in this case to explore the effects of different dts in individual NEURON sims).

import dask.bag as db # a pip installable module, usually installs without complication
def list_process(iterable_arg_dt):
    return dt_simulation_run(iterable_arg_dt)
    

cpus = multiprocessing.cpu_count -1
scattered_iterable = db.from_sequence(list,npartitions=cpus)

In the call to parallel map results of the map are implicitly gathered back onto CPU0, the main thread. So no need to manage the gathering of results back to CPU0 manually.

value = None
while dt_opt == False:
       # samples 7/8 of the search interval per 1 standard sim duration.
       gathered_list = scattered_iterable.map(list_process)
       # The following list iteration is very cheap, and should happen in 10-50 ms, in contrast to NEURON 
       # simulations, that can take seconds.
       # call the serial_function_that_computes_the_new narrow interval.
       # if dt_opt is found
       dt_opt, value = sftcnni(gathered_list)
print('dt opt found at: ',value)

Also themultiprossing class Pool, has its own map function, that behaves in the same way. I think you said you are already using multiprocessing.

I think the parallel speedup becomes the more significant after every iteration.

Second iteration:
Binary interval is narrowed to:

0.5*0.5 = 0.25

Parallel interval is narrowed to:

1/8*1/8 = 0.015625

Third iteration:
Binary interval is narrowed to:

0.5*0.5*0.5 = 0.125

Parallel interval is narrowed to:

1/8*1/8*1/8 = 0.001953125

[JustasB]

I think you're looking for the find_border function, yes?

[JustasB]

You should be able to clone the https://github.com/scrook/neuroml-db repo. But it's rather large 13GB+ and most of the relevant code is just in this folder: https://github.com/scrook/neuroml-db/tree/master/Import%20Scripts/model-importer

I have never used the code to do things in isolation (only with respect to models in NML-DB), but you might be able to get what you need from manager and manage classes.

check_install_dependencies method checks/installs dependencies needed for running NML-DB updates, but you should be able to pick and chose and install dependencies needed for your part of the code.

save_model_properties method is the primary way how I interface with the DB, get the models, and compute their properties