chore: cleanup http handlers

[achettyiitr]

Description

• Removing globals.
• Pulled out http handler related code into a separate file and added test coverage.
• Made healthTimeout and readerHeaderTimeout configurable.

Linear Ticket

• https://linear.app/rudderstack/issue/PIPE-165/http-handlers

Security

• The code changed/added as part of this pull request won't create any security issues with how the software is being used.

[achettyiitr]

@fracasula @lvrach
Should we remove goroutines from the health payload? Since we removed it from the rudder server because of security concerns.

[achettyiitr]

Also, I don't see any point in exposing it.

[fracasula]

Yes we can remove them. If we don't have the debug/pprof endpoint then we can create a task to add it.

[achettyiitr]

We have pprof endpoint.

[codecov]

Codecov Report

Patch coverage: 77.72% and project coverage change: +0.25% 🎉

Comparison is base (a7d7c74) 68.77% compared to head (11a7617) 69.02%.

❗ Current head 11a7617 differs from pull request most recent head 2403c04. Consider uploading reports for the commit 2403c04 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3767      +/-   ##
==========================================
+ Coverage   68.77%   69.02%   +0.25%     
==========================================
  Files         345      347       +2     
  Lines       51649    51643       -6     
==========================================
+ Hits        35521    35649     +128     
+ Misses      13844    13705     -139     
- Partials     2284     2289       +5     

Files Changed	Coverage Δ
warehouse/jobs/utils.go	89.28% <ø> (-1.63%)	⬇️
warehouse/utils/utils.go	85.23% <ø> (ø)
warehouse/internal/api/http.go	91.46% <50.00%> (-4.74%)	⬇️
warehouse/warehouse.go	76.64% <60.86%> (+21.50%)	⬆️
warehouse/jobs/http.go	75.72% <75.72%> (ø)
warehouse/http.go	78.10% <78.10%> (ø)
warehouse/jobs/runner.go	61.46% <100.00%> (+1.27%)	⬆️
warehouse/mode.go	100.00% <100.00%> (ø)
warehouse/multitenant/manager.go	100.00% <100.00%> (ø)

... and 9 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[fracasula]

Yes we can remove them. If we don't have the debug/pprof endpoint then we can create a task to add it.

[lvrach]

Minor, non-blocking comments

[lvrach]

Overall comment for the file:

non-5xx HTTP errors should be logged as warnings. Severity can be important in debugging, observability, and setting up alerts.

usage of structured logging will be nice as well