[enhancement] Utilize the capability of distributed tracing to support traffic recording

[sadadw1]

We currently have an idea to implement traffic recording functionality using a tracing system. Why can traffic recording be achieved using tracing?

Firstly, the technical solution for traffic recording involves logging the incoming and outgoing parameters of requests, as well as request headers, into log files within the business containers. This information is then collected by a log agent. This approach bears significant resemblance to the current architecture of tracing, and they share certain commonalities, indicating that some capabilities of tracing can be reused.

Internally within Opentelemetry, we will leverage the Nacos configuration notification mechanism. This will enable Opentelemetry to utilize Nacos configuration functionality. In the trac-etl-manager, capabilities will be provided to push configurations to Nacos.

[HawickMason]

Traffic interception involves how to intercept and collect traffics. Generally, available interception locations include：

• NIC driver
• protocol stack
• syscall
• language standard library
• communication framework (eg: rpc framework)
• business code
  For the above interception positions, the further down, the more traffic characteristics there are, and the more flexible the interception methods are.

Trace's probe's existing burying points and code enhancement capabilities can implement flexible traffic interception at the communication framework layer and business code layer, and reduce the impact on business logic in a certain flow chart, making it easier to implement record strategies. Taking RPC traffic as an example, an independent recording filter is implemented through the code enhancement of the Trace mark, and it can be conveniently determined through configuration whether to record this request or not

[wtt40122]

this is indeed a feasible solution, but we have not considered the threshold for the amount of traffic, because if the traffic is particularly high, writing files also requires a lot of overhead, and the resource IO will be very high. It's time to introduce pageCache technology