storage improvements

[frairon]

Some minor improvements to goka's default leveldb-implementation.

• no transaction during recovery
• regular offset commit during recovery

This solves:

• big memory spike when recovering huge tables
• long wait after recovery to commit the transaction
• long wait when re-opening leveldb after unclean shutdown

[andronux]

Hello!

this fix hasn't been released yet right?

We're experiencing a high spike in memory consumption when recovering huge tables (starting the app).

Thanks

[frairon]

Hey @andronux,
oh damn you're right. We were postponing the release since we've had some breaking changes and wanted to fix some bugs. to get you started though, we just pre-released the current master to v1.1.0-beta.0. There's still at least one issue for failing processors like #334, but that's quite an edge case.

[andronux]

Hey @frairon !

thank you for your quick response!

We'll go ahead and give this release a try.