Skip to content

Conversation

bzp2010
Copy link
Collaborator

@bzp2010 bzp2010 commented Aug 13, 2025

Description

ADC uses the upstream that is inlined in service, which means it needs to be written that way in adc.yaml.

services:
  - name: demo
    upstream:
      nodes:
        - host: 1.1.1.1
          port: 443
          weight: 100
    routes:
      - uris: [/*]

In the past, we did the same thing on each backend, but this caused some problems for APISIX. When the service resource was modified (because its internal upstream configuration was modified), it would cause the routing tree to be rebuilt.
According to APISIX's current routing tree reconstruction mechanism, this could cause sudden CPU spikes and result in additional traffic delays.
In Ingress Controller scenarios, changes to pod IP addresses are very common. It doesn't make sense for changes to pod IP addresses to further affect routing tree reconstruction and traffic proxying.

To avoid this issue, we have split the inline upstream in the backend implementation, replacing the upstream on the service with an upstream_id reference. This allows us to avoid the aforementioned issues.
No need to modify adc.yaml, everything is transparent and quietly completed in the background during sync and dump.

This PR implements this optimization in the apisix-standalone backend, and I also plan to implement it in apisix backend, which will be completed in another PR. API7 backend does not require this, as its control plane has already implemented the necessary functionality.

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible

@bzp2010 bzp2010 added the test/apisix-standalone Trigger the APISIX standalone test on the PR label Aug 13, 2025
@bzp2010 bzp2010 marked this pull request as draft August 13, 2025 19:12
@bzp2010 bzp2010 force-pushed the bzp/feat-bas-separate-inline-upstream-prototype branch from 8c50260 to 4a26104 Compare August 21, 2025 09:22
@bzp2010 bzp2010 marked this pull request as ready for review August 21, 2025 09:39
@bzp2010 bzp2010 changed the title feat(apisix-standalone): add prototype of separate inline upstream feat(apisix-standalone): separate inline upstream Aug 21, 2025
Comment on lines +84 to +92
newConfig[upstreamResourceKey] = [];
newConfig[upstreamResourceKey].push({
...this.fromADCUpstream(
(event.newValue as ADCSDK.Service).upstream!,
),
id: event.resourceId,
modifiedIndex: timestamp,
name: event.resourceName,
});
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
newConfig[upstreamResourceKey] = [];
newConfig[upstreamResourceKey].push({
...this.fromADCUpstream(
(event.newValue as ADCSDK.Service).upstream!,
),
id: event.resourceId,
modifiedIndex: timestamp,
name: event.resourceName,
});
newConfig[upstreamResourceKey] = [{
...this.fromADCUpstream(
(event.newValue as ADCSDK.Service).upstream!,
),
id: event.resourceId,
modifiedIndex: timestamp,
name: event.resourceName,
}];

If there are no other considerations, this change is more concise.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The newConfig is derived from the old configuration as a deep clone, not an empty object. Therefore, the key is not necessarily empty, and it is not feasible to override it directly.

const index = resources?.findIndex(
(item) => item.id === eventGeneratedId,
);
if (!isNil(index) && index != -1) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!isNil(index) && index != -1) {
if (index != -1) {

If findIndex is the only assignment method, then the value of index can only be a number.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The index is calculated by resources?.findIndex. Among ?, it is decided that if resources are undefined, this value may be undefined.

const index = resources?.findIndex(
(item) => item.id === eventGeneratedId,
);
if (!isNil(index) && index != -1) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!isNil(index) && index != -1) {
if (!isNil(index) && index != -1) {

ditto

Copy link
Contributor

@juzhiyuan juzhiyuan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the test cases and core changes and thought it would be nice to cover those cases:

  1. A service does not include inline upstream.
  2. An upstream referenced by multiple services.

But I think this can also be ensured by apisix itself?

LGTM

@bzp2010
Copy link
Collaborator Author

bzp2010 commented Aug 22, 2025

@juzhiyuan

  1. A service does not include inline upstream.

This should be possible, for example, when using some serverless plugins.
However, ADC is now not adapted to this scenario, and the upstream in the service's schema is required. Unless the user uses --no-lint to bypass the check, the input will not be accepted at all.

  1. An upstream referenced by multiple services.

ADC does not allow this use case, and the use of YAML anchor syntax will be inlined during parsing.

You may have some misunderstandings about the purpose of this PR. We have never allowed users to define upstream separately and reuse it with id, nor in this PR.

For ADC input, upstream must be inlined, which is no different from the original. This will be quietly completed behind the scenes and specific optimizations will be executed, and users should not have any perception of it.

@bzp2010 bzp2010 merged commit 6f658df into main Aug 22, 2025
22 of 26 checks passed
@bzp2010 bzp2010 deleted the bzp/feat-bas-separate-inline-upstream-prototype branch August 22, 2025 04:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test/apisix-standalone Trigger the APISIX standalone test on the PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants