Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

extract metadata changes from #29069 #12

Conversation

nfuden
Copy link
Collaborator

@nfuden nfuden commented Nov 30, 2023

metadata

jbohanon and others added 7 commits November 30, 2023 13:16
Signed-off-by: Jacob Bohanon <jacob.bohanon@solo.io>
* log grpc stream info

Signed-off-by: tyxia <tyxia@google.com>
Change API call on ext_proc to more efficient new implementation. Following the PR envoyproxy#29199, we are replacing the API call in ext_proc so it's not hashing the whole config proto on each request.

Signed-off-by: AlanDiaz <diazalan@google.com>
…proxy#29199)

This change removes the hashing of the config away from the data plane on every request to the control plane when the filter callback is created. This is accomplished by creating a wrapper object for the config which precomputes the hash and also serializes it to string ahead of time so that calls to the AsyncClientManager with that wrapper can avoid unnecessary overhead.

Signed-off-by: AlanDiaz <diazalan@google.com>
…xy#28989)

This is implementation to address issue: envoyproxy#28698.

It's a follow up PR of envoyproxy#28907

Signed-off-by: Yanjun Xiang <yanjunxiang@google.com>
@ashishb-solo
Copy link

Update to note here that I was able to get this building all extproc tests and all of them pass. I have not yet tried to build the full envoy binary, but i have reasonable confidence that it will work just fine.

The biggest issue that I have with this branch is the fact that I had to pull in a couple other cherry-picks off upstream envoy to get this to build. There were a few symbols missing in a few places that had to be pulled in. The problem with this approach is that the cherry-picks were a bit out-of-order - we merged Jacob's latest pull request in, then all the other preliminary cherry-picks that were needed to get it to build.

We can try to do better by starting fresh off the 1.27.x branch and trying to cherry-pick things in order. I think this will be more reliable as we're less likely to run into merge conflicts, and that means a lower probability of errors created by manual resolution. (Then again, don't mistake me for being one to put any faith in git's merge process as I'm sure that is perfectly capable of making errors as well. But at least we can compare both of the branches to see if there are any differences, and these assurances can hopefully give us better confidence that we did the right thing.)

tyxia and others added 9 commits November 30, 2023 20:16
* log grpc stream info

Signed-off-by: tyxia <tyxia@google.com>
…xy#28989)

This is implementation to address issue: envoyproxy#28698.

It's a follow up PR of envoyproxy#28907

Signed-off-by: Yanjun Xiang <yanjunxiang@google.com>
…proxy#29199)

This change removes the hashing of the config away from the data plane on every request to the control plane when the filter callback is created. This is accomplished by creating a wrapper object for the config which precomputes the hash and also serializes it to string ahead of time so that calls to the AsyncClientManager with that wrapper can avoid unnecessary overhead.

Signed-off-by: AlanDiaz <diazalan@google.com>
Change API call on ext_proc to more efficient new implementation. Following the PR envoyproxy#29199, we are replacing the API call in ext_proc so it's not hashing the whole config proto on each request.

Signed-off-by: AlanDiaz <diazalan@google.com>
Signed-off-by: Jacob Bohanon <jacob.bohanon@solo.io>
Introduce the ability to send attributes in the External Processing Request

---------

Signed-off-by: Jacob Bohanon <jacob.bohanon@solo.io>
@ashishb-solo
Copy link

Below were the (non-solo) commits that we cherry-picked from upstream envoy - they were necessary to resolve various functionalities on which our cherry-picks depend:

2023-08-09 4327238 ext_proc: log grpc stream byte info and upstream host (envoyproxy#28136)
2023-08-16 00309b2 Implementaion of ext_proc filter disable immediate response (envoyproxy#28989)
2023-09-06 20676d7 Optimize AsyncClientManager by moving hashing to control plane (envoyproxy#29199)
2023-09-11 9e106e5 ext_proc: performance optimization (envoyproxy#29527)

@ashishb-solo ashishb-solo merged commit dcf8bfb into release/v1.27-backportedfork Nov 30, 2023
2 checks passed
@ashishb-solo ashishb-solo deleted the extproc-1.27-from-upstreammetadata-unmerged branch November 30, 2023 21:44
ben-taussig-solo pushed a commit that referenced this pull request Jan 29, 2024
Commit Message: the probing socket is released when port migration fails. If this happens in response to an incoming packet during an I/O event, the follow socket read could cause use-after-free.

[2024-01-08 16:30:53.386][12][critical][backtrace] [./source/server/backtrace.h:104] Caught Segmentation fault, suspect faulting address 0x0
[2024-01-08 16:30:53.387][12][critical][backtrace] [./source/server/backtrace.h:91] Backtrace (use tools/stack_decode.py to get line numbers):
[2024-01-08 16:30:53.387][12][critical][backtrace] [./source/server/backtrace.h:92] Envoy version: 0/1.29.0-dev/test/DEBUG/BoringSSL
[2024-01-08 16:30:53.413][12][critical][backtrace] [./source/server/backtrace.h:96] #0: Envoy::SignalAction::sigHandler() [0x55bb876d499e]
[2024-01-08 16:30:53.413][12][critical][backtrace] [./source/server/backtrace.h:98] #1: [0x7f55fbf92510]
[2024-01-08 16:30:53.440][12][critical][backtrace] [./source/server/backtrace.h:96] #2: Envoy::Network::Utility::readPacketsFromSocket() [0x55bb875de0ef]
[2024-01-08 16:30:53.466][12][critical][backtrace] [./source/server/backtrace.h:96] #3: Envoy::Quic::EnvoyQuicClientConnection::onFileEvent() [0x55bb8663e1eb]
[2024-01-08 16:30:53.492][12][critical][backtrace] [./source/server/backtrace.h:96] #4: Envoy::Quic::EnvoyQuicClientConnection::setUpConnectionSocket()::$_0::operator()() [0x55bb8663f192]
[2024-01-08 16:30:53.518][12][critical][backtrace] [./source/server/backtrace.h:96] #5: std::__invoke_impl<>() [0x55bb8663f151]
[2024-01-08 16:30:53.544][12][critical][backtrace] [./source/server/backtrace.h:96] #6: std::__invoke_r<>() [0x55bb8663f0e2]
[2024-01-08 16:30:53.569][12][critical][backtrace] [./source/server/backtrace.h:96] #7: std::_Function_handler<>::_M_invoke() [0x55bb8663efc2]
[2024-01-08 16:30:53.595][12][critical][backtrace] [./source/server/backtrace.h:96] #8: std::function<>::operator()() [0x55bb85cb8f44]
[2024-01-08 16:30:53.621][12][critical][backtrace] [./source/server/backtrace.h:96] #9: Envoy::Event::DispatcherImpl::createFileEvent()::$_5::operator()() [0x55bb8722560f]
[2024-01-08 16:30:53.648][12][critical][backtrace] [./source/server/backtrace.h:96] #10: std::__invoke_impl<>() [0x55bb872255c1]
[2024-01-08 16:30:53.674][12][critical][backtrace] [./source/server/backtrace.h:96] #11: std::__invoke_r<>() [0x55bb87225562]
[2024-01-08 16:30:53.700][12][critical][backtrace] [./source/server/backtrace.h:96] #12: std::_Function_handler<>::_M_invoke() [0x55bb872253e2]
[2024-01-08 16:30:53.700][12][critical][backtrace] [./source/server/backtrace.h:96] #13: std::function<>::operator()() [0x55bb85cb8f44]
[2024-01-08 16:30:53.726][12][critical][backtrace] [./source/server/backtrace.h:96] #14: Envoy::Event::FileEventImpl::mergeInjectedEventsAndRunCb() [0x55bb872358ec]
[2024-01-08 16:30:53.752][12][critical][backtrace] [./source/server/backtrace.h:96] #15: Envoy::Event::FileEventImpl::assignEvents()::$_1::operator()() [0x55bb87235ed1]
[2024-01-08 16:30:53.778][12][critical][backtrace] [./source/server/backtrace.h:96] #16: Envoy::Event::FileEventImpl::assignEvents()::$_1::__invoke() [0x55bb87235949]
[2024-01-08 16:30:53.804][12][critical][backtrace] [./source/server/backtrace.h:96] #17: event_persist_closure [0x55bb87fab72b]
[2024-01-08 16:30:53.830][12][critical][backtrace] [./source/server/backtrace.h:96] #18: event_process_active_single_queue [0x55bb87faada2]
[2024-01-08 16:30:53.856][12][critical][backtrace] [./source/server/backtrace.h:96] #19: event_process_active [0x55bb87fa56c8]
[2024-01-08 16:30:53.882][12][critical][backtrace] [./source/server/backtrace.h:96] #20: event_base_loop [0x55bb87fa45cc]
[2024-01-08 16:30:53.908][12][critical][backtrace] [./source/server/backtrace.h:96] #21: Envoy::Event::LibeventScheduler::run() [0x55bb8760a59f]
Risk Level: low
Testing: new unit test
Docs Changes: N/A
Release Notes: Yes
Platform Specific Features: N/A

Signed-off-by: Dan Zhang <danzh@google.com>
Co-authored-by: Dan Zhang <danzh@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants