Memory Cost of Deep Nets under Different Allocations
This folder contains a script to show the memory cost of different allocation strategies, discussed in Note on Memory Optimization.
We use inception-bn as an example, with batch size of 32.
How to See the cost
The possible options are gathered together in the Makefile.
Type the following command to see the allocation cost. Look for the
Final message Total x MB allocated
- Shows the cost without any optimization.
- Shows the cost with inplace optimization.
- Shows the cost with memory allocating algorithm for sharing.
- Shows the cost of memory allocation with both inplace and sharing optimization.
- Shows the cost of when we only want to run forward pass.
- You can change the symbol in the inception_memcost.py to the net you interested in.
- You will need to install mxnet or type make on the root folder before use the script.
- The estimation is only on space cost of intermediate node.
- The cost of temporal workspace is not estimated, so you will likely need more memory when running real nets.
- The estimation does real allocation on CPU, the plan is the same on GPU.