Realtime node cannot start back up if there is a "merged" directory on disk

[cheddar]

The realtime nodes periodically merge data and store it in a local directory for pushing up to deep storage. When these directories are left lying around, the screw up the start up logic with exceptions like:

[2013-04-17 15:58:43,178] ERROR RuntimeException aborted realtime processing[rb_flow]: {class=com.metamx.druid.realtime.RealtimeManager, exceptionType=class java.lang.NumberFormatException, exceptionMessage=For input string: "merged"} (co
java.lang.NumberFormatException: For input string: "merged"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:492)
at java.lang.Integer.parseInt(Integer.java:527)
at com.metamx.druid.realtime.plumber.RealtimePlumberSchool$1.bootstrapSinksFromDisk(RealtimePlumberSchool.java:377)
at com.metamx.druid.realtime.plumber.RealtimePlumberSchool$1.startJob(RealtimePlumberSchool.java:188)
at com.metamx.druid.realtime.RealtimeManager$FireChief.run(RealtimeManager.java:159)

We need to fix this in two ways:

1. Make the restart logic resilient to this stuff
2. Store the merged directories somewhere else so that they don't get in the way in the first place

[sixtus]

Not involved, just trying to keep this up to date: There is a pull request now: https://github.com/metamx/druid/pull/158

[quode]

Thanks sixtus. I forgot to link pull request #158 to this bug.

[cheddar]

Merged. Thanks!