Stack overflow for large chains of shared_ptr (or smart pointers in general)

[AzothAmmo]

It is possible to encounter a stack overflow when serializing data structures that utilize a large amount of pointers. Our implementation essentially utilizes a recursive depth first search to serialize pointers which can easily overflow the stack if the number of elements is large.

This was encountered in a neural network library where shared pointers are used to track connections between neurons.

To fix this: pointer serialization needs to be transformed into an iterative algorithm probably with a local data structure holding lambdas to perform the serialization.

[AzothAmmo]

See discussion https://groups.google.com/forum/#!topic/cerealcpp/_eWdB1kV9QI

[bloodcarter]

Any news on this bug? Looks like I've run into it too :(

[AzothAmmo]

I've never taken the time to engineer a solution and have been using a workaround of setting my stack size to be larger (using ulimit on linux). I unfortunately do not have a lot of time to devote to cereal at the moment but hopefully will be able to make a solid push on various issues in the future. Solving this, as I mentioned in the original posting, involves a fairly significant re-write of how we perform serialization for recursive data structures.

[AzothAmmo]

So I've run into this again myself and it is essentially blocking my own work. I did some searching and it's been noted as a problem for boost as well.

Unfortunately there really isn't a good way to resolve this - I don't think it is possible to create an iterative serialization pipeline within the framework cereal provides. For now I suggest working around this as best as possible by restructuring your data or deferring parts of your serialization to avoid deep recursion.

[AzothAmmo]

I have an idea for a solution for this. It's not the ideal turn serialization into an iteration which would completely solve the problem, but it's a nice compromise.

I'm thinking of adding something like CEREAL_DEFER, which will cause the serialization to be skipped and done at a later time by calling some other function, or possibly passing something to the archive (like a fake serialization item that does all deferments).

This would work well for things like graphs where you have a lot of nodes and then non-owning connections between the nodes. Normally, with shared pointers, the first item serialized would do a DFS to repopulate every node reachable. With this method, you would defer your connection serialization and first serialize the nodes themselves. When it came time to do the connections, the nodes would already exist so there would be no recursion.

[AzothAmmo]

So cereal::defer effectively solves this if you can structure your serialization or code appropriately. In my use case, for example, I have a large neural network defined by neurons and connections between them - basically a graph structure. I defer the serialization of the connections until the nodes themselves have been serialized, thus preventing deep recursion on traversing the graph.

I still need to clean up and document this feature, but I'm really liking it so far.

Note to self: breaks if an NVP is nested in a defer.

[AzothAmmo]

Turns out there likely is a decent solution to go iterative but it would turn serialization into a two-stage process. It would be a lot of work overall and come with various pros and cons, so it's something I'll consider in a future release. For now I think the defer concept is a good substitute.