Handle truncated translog gracefully

[dakrone]

We used to handle truncated translogs in a better manner (assuming that
the node was killed halfway through writing an operation and discarding
the last operation). This brings back that behavior by catching an
EOFException during the stream reading and throwing a
TruncatedTranslogException which can be safely ignored in
IndexShardGateway.

Fixes #9699

[kimchy]

@dakrone this looks good, I wonder if in the test we should write a translog and then munge it to remove the last few bytes to simulate transactions? that way we don't have a file that might eventually not support backward comp. on written ops?

I think its a good change since it restores the previous behavior, would be good to work on the per op better translog handling as well eventually that will make this more resilient (can't find the issue now)

[dakrone]

@kimchy sounds good, I added another test that tests truncated translog files (for a non-pre-generated translog)

[kimchy]

@dakrone it looks great, I would consider adding another test, that reads the data in a similar manner that the gateway does (reading it one by one, not based on locations), and seeing that we get a truncated exception once we work no a truncated translog? other than that, LGTM