feat: add health Probe

[susanshi]

Description

fixes #977

What this PR does / why we need it:

• add probe to deployment files
• allow probe port to be configuration ( code changes required)

Tests:
validated default probe port
validated non default port
validated probe events appear in pod events

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):

Fixes #

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Helm Chart Change (any edit/addition/update that is necessary for changes merged to the main branch)
• This change requires a documentation update

How Has This Been Tested?

Tests:
validated default probe port
validated non default port
validated probe events appear in pod events ( only failure appears in event log)

Checklist:

• Does the affected code have corresponding tests?
• Are the changes documented, not just with inline documentation, but also with conceptual documentation such as an overview of a new feature, or task-based documentation like a tutorial? Consider if this change should be announced on your project blog.
• Does this introduce breaking changes that would require an announcement or bumping the major version?
• Do all new files have appropriate license header?

Post Merge Requirements

• MAINTAINERS: manually trigger the "Publish Package" workflow after merging any PR that indicates Helm Chart Change

[codecov]

Codecov Report

Patch coverage: 16.66% and project coverage change: -0.02% ⚠️

Comparison is base (40834b6) 58.32% compared to head (4a6683b) 58.31%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1058      +/-   ##
==========================================
- Coverage   58.32%   58.31%   -0.02%     
==========================================
  Files          93       93              
  Lines        5543     5546       +3     
==========================================
+ Hits         3233     3234       +1     
- Misses       1994     1996       +2     
  Partials      316      316              

Files Changed	Coverage Δ
httpserver/server.go	43.80% <ø> (ø)
pkg/manager/manager.go	3.31% <0.00%> (-0.04%)	⬇️
cmd/ratify/cmd/serve.go	39.06% <50.00%> (+0.96%)	⬆️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[binbin-li]

lgtm, just 2 minor comments

[susanshi]

Hi @binbin-li , this is ready for another round of review. The HA test was failing due to something else was already running at 9090, i ve updated the default port to 9099