Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Carry static page in Bytes structure to avoid expensive allocation per request #4528

Merged
merged 4 commits into from
Jan 24, 2024
Merged

Carry static page in Bytes structure to avoid expensive allocation per request #4528

merged 4 commits into from
Jan 24, 2024

Conversation

xuorig
Copy link
Contributor

@xuorig xuorig commented Jan 20, 2024 •
edited

I noticed the StaticPageLayer causes a non-insignificant amount of memory to be allocated on every request. The service stack gets cloned on every request, and so does the CheckpointService created by the StaticPageLayer, even on non-static-page requests.

This is only expensive because templates are pre-rendered and kept on the service struct itself, this relatively large string then gets cloned on every request.

This pull request addresses this issue by instead moving the rendering only on static page requests. Of course, this carries a cost at every request, but only on less important (relative to service router requests) requests. Instead of the full string, a small enum is cloned that contains the necessary configuration to render the pages.

To @Geal 's suggestion, I've update the PR to store the rendered template in a Bytes struct instead.

I did not investigate a way to cache those outside the services yet, although it could certainly be possible. Looking to get some feedback on the report / idea before I investigate further. Thanks!

@router-perf
Copy link

router-perf bot commented Jan 20, 2024

CI performance tests

  • events_big_cap_high_rate - Stress test for events with a lot of users, deduplication enabled and high rate event with a big queue capacity
  • events_without_dedup - Stress test for events with a lot of users and deduplication DISABLED
  • events - Stress test for events with a lot of users and deduplication ENABLED
  • large-request - Stress test with a 1 MB request payload
  • step - Basic stress test that steps up the number of users over time
  • xlarge-request - Stress test with 10 MB request payload
  • reload - Reload test over a long period of time at a constant rate of users
  • no-graphos - Basic stress test, no GraphOS.
  • events_callback - Stress test for events with a lot of users and deduplication ENABLED in callback mode
  • xxlarge-request - Stress test with 100 MB request payload
  • step-jemalloc-tuning - Clone of the basic stress test for jemalloc tuning
  • events_without_dedup_callback - Stress test for events with a lot of users and deduplication DISABLED using callback mode
  • const - Basic stress test that runs with a constant number of users
  • events_big_cap_high_rate_callback - Stress test for events with a lot of users, deduplication enabled and high rate event with a big queue capacity using callback mode

@Geal
Copy link
Contributor

Geal commented Jan 22, 2024

right, reallocating this on every request does not make much sense. I'm wondering, could it be better to generate both pages once, when creating the layers, and storing that in a Bytes structure? It will be refcounted, and there's an easy transformation to a response body

@xuorig
Copy link
Contributor Author

xuorig commented Jan 22, 2024

Great idea @Geal, let me give it a try today.

@xuorig xuorig changed the title Render static pages on demand rather than carrying String on service Carry static page in Bytes structure to avoid expensive allocation per request Jan 22, 2024
@xuorig
Copy link
Contributor Author

xuorig commented Jan 22, 2024

@Geal Modified to use Bytes. Not sure what is going on with the CI checks. added a changelog too.

@Geal Geal requested a review from a team January 23, 2024 09:17
goto-bus-stop
goto-bus-stop approved these changes Jan 23, 2024
View reviewed changes
Copy link
Member

@goto-bus-stop goto-bus-stop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sweet, nice find

@Geal Geal merged commit 6d55932 into apollographql:dev Jan 24, 2024
12 of 13 checks passed
This was referenced Feb 1, 2024
Closed
Merged
Geal added a commit that referenced this pull request Feb 5, 2024
@Geal @shorgi
@o0Ignition0o @abernix
prep release: v1.39.0 (#4597)
fadd19b 
## 🚀 Features

### Specify Trace ID Formatting ([PR
#4530](#4530))

This adds the ability to specify the format of the trace ID in the
response headers of the supergraph service.

An example configuration making use of this feature is shown below:
```yaml
telemetry:
  apollo:
    client_name_header: name_header
    client_version_header: version_header
  exporters:
    tracing:
      experimental_response_trace_id:
        enabled: true
        header_name: trace_id
        format: decimal # Optional, defaults to hexadecimal
```

If the format is not specified, then the trace ID will continue to be in
hexadecimal format.

By [@nicholascioli](https://github.com/nicholascioli) in
#4530

### Introduce support for progressive `@override` ([PR
#4521](#4521))

The change brings support for progressive `@override`, which allows
dynamically overriding root fields and entity fields in the schema. This
feature is enterprise only and requires a license key to be used.

A new `label` argument is added to the `@override` directive in order to
indicate the field is dynamically overridden. Labels can come in two
forms:
1) String matching the form `percent(x)`: The router resolves these
labels based on the `x` value. For example, `percent(50)` will route 50%
of requests to the overridden field and 50% of requests to the original
field.
2) Arbitrary string matching the regex `^[a-zA-Z][a-zA-Z0-9_-:./]*$`:
These labels are expected to be resolved externally via coprocessor. A
coprocessor a supergraph request hook can inspect and modify the context
of a request in order to inform the router which labels to use during
query planning.

Please consult the docs for more information on how to use this feature
and how to implement a coprocessor for label resolution.

By [@trevorscheer](https://github.com/TrevorScheer) in
#4521

### Add a new selector to get all baggage key values in span attributes
([Issue #4425](#4425))

If you have several baggage items and would like to add all of them
directly as span attributes, for example `baggage: my_item=test,
my_second_item=bar` will add 2 span attributes `my_item=test` and
`my_second_item=bar`.

Example of configuration:

```yaml
telemetry:
  instrumentation:
    spans:
      router:
        attributes:
          baggage: true
```

By [@bnjjj](https://github.com/bnjjj) in
#4537

### Create a trace during router creation and plugin initialization
([Issue #4472](#4472))

When the router starts or reload, it will now generate a trace with
spans for query planner creation, schema parsing, plugin initialization
and request pipeline creation. This will help in debugging any issue
during startup, especially in plugins creation

By [@Geal](https://github.com/Geal) in
#4480

### Add static attribute on specific span in telemetry settings ([Issue
#4561](#4561))

Example of configuration:

```yaml
telemetry:
  instrumentation:
    spans:
      router:
        attributes:
          "my_attribute": "constant_value"
      supergraph:
        attributes:
          "my_attribute": "constant_value"
      subgraph:
        attributes:
          "my_attribute": "constant_value"
```

By [@bnjjj](https://github.com/bnjjj) in
#4566

## 🐛 Fixes

### Order HPA target so that kubernetes does not rewrite ([Issue
#4435](#4435))

This update addresses an OutOfSync issue in ArgoCD applications when
Horizontal Pod Autoscaler (HPA) is configured with both memory and CPU
limits.
Previously, the live and desired manifests within Kubernetes were not
consistently sorted, leading to persistent OutOfSync states in ArgoCD.
This change implements a sorting mechanism for HPA targets within the
Helm chart, ensuring alignment with Kubernetes' expected order.
This fix proactively resolves the sync discrepancies while using HPA,
circumventing the need to wait for Kubernetes' issue resolution
(kubernetes/kubernetes#74099).

By [@cyberhck](https://github.com/cyberhck) in
#4436

### Reactivate log events in traces ([PR
#4486](#4486))

This fixes a regression introduced in #2999, where events were not sent
with traces anymore due to too aggressive sampling

By [@Geal](https://github.com/Geal) in
#4486

### Fix inconsistency in environment variable parsing for telemetry
([Issue
#3203](https://github.com/apollographql/router/issues/ISSUE_NUMBER))

Previously, the router would complain when using the rover
recommendation of `APOLLO_TELEMETRY_DISABLED=1` environment
variable. Now any non-falsey value can be used, such as 1, yes, on,
etc..

By [@nicholascioli](https://github.com/nicholascioli) in
#4549

### Store static pages in `Bytes` structure to avoid expensive
allocation per request ([PR
#4528](#4528))

The `CheckpointService` created by the `StaticPageLayer` caused a
non-insignificant amount of memory to be allocated on every request. The
service stack gets cloned on every request, and so does the rendered
template.

The template is now stored in a `Bytes` struct instead which is cheap to
clone.

By [@xuorig](https://github.com/xuorig) in
#4528

### Fix header propagation issues ([Issue
#4312](#4312)), ([Issue
#4398](#4398))

This fixes two header propagation issues:
* if a client request header has already been added to a subgraph
request due to another header propagation rule, then it is only added
once
* `Accept`, `Accept-Encoding` and `Content-Encoding` were not in the
list of reserved headers that cannot be propagated. They are now in that
list because those headers are set explicitely by the Router in its
subgraph requests

There is a potential regression: if a router deployment was accidentally
relying on header propagation to compress subgraph requests, then it
will not work anymore because `Content-Encoding` is not propagated
anymore. Instead it should be set up from the `traffic_shaping` section
of the Router configuration:

```yaml
traffic_shaping:
  all:
    compression: gzip
  subgraphs: # Rules applied to requests from the router to individual subgraphs
    products:
      compression: identity
```

By [@Geal](https://github.com/Geal) in
#4535

## 🧪 Experimental

### Move cacheability metrics to the entity cache plugin ([Issue
#4253](#4253))

The metric was generated in the telemetry plugin before, but it was not
very convenient to keep it there. This adds more configuration:
- enable or disable the metrics
- set the metrics storage TTL (default is 60s)
- the metric's typename attribute is disabled by default. Activating it
can greatly increase the cardinality

This also includes some cleanup and performance improvements

By [@Geal](https://github.com/Geal) in
#4469

---------

Co-authored-by: Edward Huang <edward.huang@apollographql.com>
Co-authored-by: Jeremy Lempereur <jeremy.lempereur@iomentum.com>
Co-authored-by: Jesse Rosenberger <git@jro.cc>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Geal Geal Geal approved these changes

@goto-bus-stop goto-bus-stop goto-bus-stop approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@xuorig @Geal @goto-bus-stop