[FLINK-4000] Checkpoint dictionaries null after taskmgr failures

[rekhajoshm]

Fix for exception during job restart after task mgr failure, at which point restoreState fails as checkpoint dictionaries can be null.

[asavartsov]

This kind of check is might be useless and probably wouldn't fix the issue. My debugging shows that the list idsProcessedButNotAcknowledged is null on recovery, not checkpoints itself. This list is initialized in open method, but somehow it doesn't get called in such scenario.

[rekhajoshm]

I agree @asavartsov , that was a quick look and was working to reproduce.Does the updated make sense? thank you.

[asavartsov]

No, it does not make any sense and even makes things worse, sorry.

[rekhajoshm]

@asavartsov Ok. Please let me know how you propose to solve this? thanks

[asavartsov]

Take a look at my pull request at #2062

[rekhajoshm]

aha, in one of my intermediate runs had done just initializing idsProcessedButNotAcknowledged and retain pendingCheckpoints , but in last run changed it calling open() :-( . makes sense @asavartsov. closing. thanks