[FLINK-39761][runtime] Set 'Connection: close' header when sending '304 Not Modified' responses

[nattilabalint]

What is the purpose of the change

Fix the proxy connection pool poisoning (putting back closed connections to the pool) on 304 Not Modified responses.
This is happening, because Flink severe the connections from its end when respond with HTTP_304, without explicitly setting the Connection: close header on the response.

Brief change log

The Connection: close header added to the response.

Verifying this change

This change added tests and can be verified as follows:

• Added unit test to verify the Connection: close header
• The proxy connection pool poisoning related errors disappeared (repro env: Flink dashboard behind Knox, browser reload sends 'If-Modified-Since' header for cached static assets (.js, .css), Flink respond with HTTP_304, no failed subsequent GET failures anymore)

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Gemini 3.5

[flinkbot]

CI report:

• 3b2489f Azure: FAILURE

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[spuru9]

Can you take a look at the comments. Thanks.

[spuru9]

Test uses Mockito, which Flink forbids via checkstyle
https://flink.apache.org/how-to-contribute/code-style-and-quality-common/#avoid-mockito---use-reusable-test-implementations

[spuru9]

The test correctly targets JUnit 5 (org.junit.jupiter), but two conventions are off for new Flink test code:

1. Drop public on the class and the @Test method — JUnit 5 doesn't require it and Flink's style drops it.
2. Use AssertJ, not Jupiter Assertions. Flink standardizes on AssertJ (assertThat(...)).

[ferenc-csaky]

@flinkbot run azure

[ferenc-csaky]

Change LGTM, from a quick peek CI failure seems to be orthogonal, let's give that another try.

Before merge, pls. also open backport PRs for the fix versions defined in the corresponding jira.

[spuru9]

Thanks for addressing the comments. LGTM.