[FEATURE] Harvester supports event log

[w13915984028]

Is your feature request related to a problem? Please describe.

Requirement: #578

Describe the solution you'd like

Collect (kubernetes) event log by default.

Send event log to loki by default, which can be queried by embedded grafana.

Support send event logs to general log servers.

Describe alternatives you've considered

Additional context

HEP PR:
#2684
HEP 578 event and audit log [CI SKIP] #2684

[harvesterhci-io-github-bot]

Pre Ready-For-Testing Checklist

• If labeled: require/HEP Has the Harvester Enhancement Proposal PR submitted?
  The HEP PR is at: HEP 578 event and audit log [CI SKIP] #2684 HEP 578 event and audit log

* [ ] Where is the reproduce steps/test steps documented? The reproduce steps/test steps are at:

• Is there a workaround for the issue? If so, where is it documented?
  The workaround is at:

• Have the backend code been merged (harvester, harvester-installer, etc) (including backport-needed/*)?
  The PR is at: Feat578 audit and event log harvester-installer#333

  • Does the PR include the explanation for the fix or the feature?

  • Does the PR include deployment change (YAML/Chart)? If so, where are the PRs for both YAML file and Chart?
    The PR for the YAML change is at: Feat578 audit and event log harvester-installer#333
    The PR for the chart change is at: Feat578 audit and event log harvester-installer#333

* [ ] **If labeled: area/ui** Has the UI issue filed or ready to be merged? The UI issue/PR is at:

• If labeled: require/doc, require/knowledge-base Has the necessary document PR submitted or merged?
  The documentation/KB PR is at:

• If NOT labeled: not-require/test-plan Has the e2e test plan been merged? Have QAs agreed on the automation test case? If only test case skeleton w/o implementation, have you created an implementation issue?
  • The automation skeleton PR is at:
  • The automation test case PR is at:

* [ ] **If the fix introduces the code for backward compatibility** Has a separate issue been filed with the label `release/obsolete-compatibility`? The compatibility issue is filed at:

[harvesterhci-io-github-bot]

Automation e2e test issue: harvester/tests#504

[w13915984028]

FOR TEST:

In v1.1.0, test logging/event/audit output type of: Graylog, Splunk, they are required by customers;
webhook server is for quick validation.

Set output manually :
How to send log to graylog:
https://github.com/w13915984028/harvester-develop-summary/blob/main/integrate-harvester-logging-with-log-servers.md

Test logging, audit, event with webhook server
https://github.com/w13915984028/harvester-develop-summary/blob/main/test-log-event-audit-with-webhook-server.md

Set output in UI:
#2646
[FEATURE] Config logging in Harvester Dashboard

#2749
[FEATURE] Harvester UI support config (cluster) flow & output for kube-audit log and event log

[lanfon72]

Verified this feature has been implemented.

Test Information

• Environment: qemu/KVM 3 nodes
• Harvester Version: master-250f41e4-head
• ui-source Option: Auto

Verify Steps:

1. Install Graylog via docker[^1]
2. Install Harvester with any nodes
3. Login to Dashboard then navigate to Monitoring & Logging/Logging
4. Create Cluster Output with following:
   • Name: gelf-evts
   • Type: Logging/Event
   • Output: GELF
   • Target: <Graylog_IP>, <Graylog_Port>, <UDP>
5. Create Cluster Flow with following:
   • Name: gelf-flow
   • Type of Matches: Event
   • Cluster Outputs: gelf-evts
6. Create an Image for VM creation
7. Create a vm vm1 and start it
8. Login to Graylog dashboard then navigate to search
9. Select update frequency
   
10. New logs should be posted continuously.

code snippets to setup Graylog

docker run --name mongo -d mongo:4.2.22-rc0
sysctl -w vm.max_map_count=262145
docker run --name elasticsearch -p 9200:9200 -p 9300:9300 -e xpack.security.enabled=false  -e node.name=es01 -it docker.elastic.co/elasticsearch/elasticsearch:6.8.23
docker run --name graylog --link mongo --link elasticsearch -p 9000:9000 -p 12201:12201 -p 1514:1514 -p 5555:5555 -p 12202:12202 -p 12202:12202/udp -e GRAYLOG_PASSWORD_SECRET="Graypass3WordMor!e" -e GRAYLOG_ROOT_PASSWORD_SHA2=899e9793de44cbb14f48b4fce810de122093d03705c0971752a5c15b0fa1ae03   -e GRAYLOG_HTTP_EXTERNAL_URI="http://127.0.0.1:9000/"  -d graylog/graylog:4.3.5

• Login to Graylog dashboard by the URL http://<server_ip>:9000/ with admin/ROOT_PASSWORDa1
• Navigate to System/Inputs then select input GELF UDP, update the port to 12202