update element maps in batches #27

Closed
ddale opened this Issue May 17, 2011 · 1 comment

1 participant

@ddale
Praxes Developers member

explore the possibility of speeding up IO:

many_results = pool.map(...)

#for each element, maptype:
res = [i for i in many_results if i is calcium]
update(res)

@ddale
Praxes Developers member

Ended up taking a different approach, using a results proxy to store and update numpy arrays in memory. Those arrays are not written to file until data processing has completed.

@ddale ddale closed this Jan 3, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment