[SPARK-49930][SS] Ensure that socket updates are flushed on exception from the python worker

[anishshri-db]

What changes were proposed in this pull request?

Ensure that socket updates are flushed on exception from the python worker

Why are the changes needed?

Without this, updates to the socket from the python worker are not delivered to the jvm side

Does this PR introduce any user-facing change?

No

How was this patch tested?

Existing tests were failing on a different Python version and pass after

Run completed in 1 minute, 13 seconds.
Total number of tests run: 8
Suites: completed 1, aborted 0
Tests: succeeded 8, failed 0, canceled 0, ignored 0, pending 0
All tests passed.

Was this patch authored or co-authored using generative AI tooling?

No

[anishshri-db]

cc - @HeartSaVioR @WweiL - PTAL, thx !

[HyukjinKwon]

Merged to master.