Reduce memory consumption in node creation with many threads

[hakonsbm]

Creating nodes of different models while using many threads uses an excessive amount of memory. For example, creating 10 populations of 5000 nodes each and different models for each population:

Threads	Memory [MB]
1	135
256	2255

This is because node creation relies on memory management with sli::pool:

nest-simulator/nestkernel/model.h

Line 253 in d2e4974

std::vector< sli::pool > memory_;

By replacing the sli::pool with an std::vector, the memory consumption is kept at a more manageable level:

Threads	Memory [MB]
1	129
256	306

[heplesser]

@hakonsbm That is an impressive reduction in memory use. It would be interesting to see construction and simulation times for realistic network sizes as well. And could memory consumption be reduced even further using BlockVector without noticeable performance loss?

[hakonsbm]

@heplesser For connection and simulation the memory_ vector is actually not used. The node pointer is added to local_nodes_, which is the one used for connection and simulation:

nest-simulator/nestkernel/node_manager.cpp

Lines 252 to 261 in d2e4974

	Node* node = model.allocate( t );
	node->set_node_id_( node_id );
	node->set_nc_( nc_ptr );
	node->set_model_id( model.get_model_id() );
	node->set_thread( t );
	node->set_vp( kernel().vp_manager.thread_to_vp( t ) );
	node->set_local_device_id( num_thread_local_devices_[ t ] - 1 );
	node->set_initialized();
	local_nodes_[ t ].add_local_node( *node );

Therefore, the connection and simulation times should be unaffected, which a run of hpc_benchmark with a scale of 20 supports.

Using BlockVector instead of std::vector actually uses more memory (696 MB with BlockVector vs 306 MB with std::vector). That's because BlockVector initializes 1024 elements when created, and that is done 256 times (once for each thread) for every model, leading to a big overhead.

[heplesser]

@hakonsbm Good to know that building and simulation time is not affected. But I think the memory_ is used in the sense that this is where the neuron objects are stored, so if memory layout had an effect on performance, we might have seen changes. But given that vector stores objects contiguously, this is presumably the optimal memory layout.

[heplesser]

Looks generally good, just a few comments/questions.

[heplesser]

Comment is outdated.

[heplesser]

Are allocate() and allocate_() actually good names? Wouldn't create or clone be more readable?

[heplesser]

Could it make sense to keep reserve_additional to avoid unnecessary automatic vector resizings?

[jougs]

I'm fine with the changes here. Many thanks!

More fundamentally, I'm wondering if we wouldn't be better of with simple thread-local vectors of nodes of a certain type in the GenericModel instead of packing them into custom pool-based allocators that have never been really tested or designed for performance... But that's probably for another time.