Investigate memory usage (caching vs memory optimization)

[dirkjanm]

In large networks the memory usage can grow quite fast, creating trouble on drop devices such as raspberry Pis.
There are several caches in the code, which could probably be optimized.

Some things to look at:

• Which cache grows fastest on larger networks
• Can stuff be removed from the cache once it's likely no longer useful
• Can the amount of detail in the cache be reduced
• Are objects GC'd properly when not cached.

[dirkjanm]

caches have been mostly optimized, anything else would introduce major network overhead, so closing this