[improve] [proxy] PIP-395: Add Proxy configuration to support configurable response headers for http reverse-proxy

[rdhabalia]

PIP: #23648

Motivation

Pulsar Proxy has a support to use it as HTTP reverse proxy to access Broker’s admin API and can also be extended to act as a reverse proxy other HTTP endpoints. Sometimes, it is very crucial to add customizable headers into the HTTP response returned by Proxy to enhance the security experience while using Proxy over HTTP.

Response headers in a reverse HTTP proxy are critical for maintaining and enhancing the security of the web applications and services behind the proxy. These headers act as a first line of defense, hardening the web server infrastructure and protecting clients from common web vulnerabilities.

For example, when implementing a reverse HTTP proxy, security headers such as Referrer-Policy, X-Content-Type-Options, Strict-Transport-Security, X-Content-Type-Options, etc., are useful to prevent security attacks like clickjacking, MIME-sniffing, data leakage, and more. So, such headers play a crucial role in enhancing the security posture of proxy infrastructure.

Therefore, we would like to add support into Pulsar Proxy where users can add custom response headers by passing them into the configuration. This PIP will add this support by adding a new configuration called proxyHttpResponseHeadersJson where user can pass multiple headers with key-value map into the json format. Proxy server will retrieve headers from this configuration and pass it as response headers for every http request when user wants to use Pulsar Proxy as an HTTP reverse proxy.

Modifications

Add a new configuration proxyHttpResponseHeadersJson to the Proxy configuration.
eg:

proxyHttpResponseHeadersJson=`{"header1":"value1","header2":"value2"}`

Verifying this change

• Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

• Added integration tests for end-to-end deployment with large payloads (10MB)
• Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository:

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 85.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 74.19%. Comparing base (bbc6224) to head (75afd57).
Report is 911 commits behind head on master.

Files with missing lines	Patch %	Lines
...java/org/apache/pulsar/proxy/server/WebServer.java	85.00%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #23649      +/-   ##
============================================
+ Coverage     73.57%   74.19%   +0.61%     
+ Complexity    32624    31871     -753     
============================================
  Files          1877     1853      -24     
  Lines        139502   143840    +4338     
  Branches      15299    16339    +1040     
============================================
+ Hits         102638   106716    +4078     
+ Misses        28908    28725     -183     
- Partials       7956     8399     +443     

Flag	Coverage Δ
inttests	26.78% <50.00%> (+2.19%)	⬆️
systests	23.24% <50.00%> (-1.08%)	⬇️
unittests	73.70% <85.00%> (+0.85%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...apache/pulsar/proxy/server/ProxyConfiguration.java	95.45% <ø> (-1.70%)	⬇️
...java/org/apache/pulsar/proxy/server/WebServer.java	84.16% <85.00%> (-3.11%)	⬇️

... and 1037 files with indirect coverage changes