Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Caching: Add feature toggle for memory efficient cache payload serialization #76145

Merged
merged 8 commits into from
Oct 12, 2023

Conversation

mmandrus
Copy link
Contributor

@mmandrus mmandrus commented Oct 6, 2023

What is this feature?

Creates a new feature toggle used by the enterprise caching feature.

# Conflicts:
#	docs/sources/setup-grafana/configure-grafana/feature-toggles/index.md
#	packages/grafana-data/src/types/featureToggles.gen.ts
#	pkg/services/featuremgmt/registry.go
#	pkg/services/featuremgmt/toggles_gen.csv
#	pkg/services/featuremgmt/toggles_gen.go
@mmandrus mmandrus added this to the 10.2.x milestone Oct 9, 2023
@mmandrus mmandrus added add to changelog no-backport Skip backport of PR labels Oct 9, 2023
@mmandrus mmandrus changed the title Caching: Add feature toggle for smart cache serialization Caching: Add feature toggle for memory efficient cache payload serialization Oct 9, 2023
Comment on lines 30 to 32
| `cloudwatchNewRegionsHandler` | Refactor of /regions endpoint, no user-facing changes | Yes |
| `nestedFolderPicker` | Enables the new folder picker to work with nested folders. Requires the nestedFolders feature flag | Yes |
| `accessTokenExpirationCheck` | Enable OAuth access_token expiration check and token refresh using the refresh_token | |
Copy link
Contributor

@knylander-grafana knylander-grafana Oct 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correcting some minor formatting issues.

Suggested change
| `cloudwatchNewRegionsHandler` | Refactor of /regions endpoint, no user-facing changes | Yes |
| `nestedFolderPicker` | Enables the new folder picker to work with nested folders. Requires the nestedFolders feature flag | Yes |
| `accessTokenExpirationCheck` | Enable OAuth access_token expiration check and token refresh using the refresh_token | |
| `cloudwatchNewRegionsHandler` | Refactor of `/regions` endpoint, no user-facing changes | Yes |
| `nestedFolderPicker` | Enables the new folder picker to work with nested folders. Requires the `nestedFolders` feature toggle | Yes |
| `accessTokenExpirationCheck` | Enable OAuth access_token expiration check and token refresh using the `refresh_token` | |

| `prometheusConfigOverhaulAuth` | Update the Prometheus configuration page with the new auth component | Yes |
| `newBrowseDashboards` | New browse/manage dashboards UI | Yes |
| `cloudWatchWildCardDimensionValues` | Fetches dimension values from CloudWatch to correctly label wildcard dimensions | Yes |
| `cachingOptimizeSerializationMemoryUsage` | If enabled, the caching backend will gradually serialize query responses for the cache, comparing against the configured `[caching]max_value_mb` value as it goes, which can help prevent Grafana from running out of memory while attempting to cache very large query responses | |
Copy link
Contributor

@knylander-grafana knylander-grafana Oct 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| `cachingOptimizeSerializationMemoryUsage` | If enabled, the caching backend will gradually serialize query responses for the cache, comparing against the configured `[caching]max_value_mb` value as it goes, which can help prevent Grafana from running out of memory while attempting to cache very large query responses | |
| `cachingOptimizeSerializationMemoryUsage` | If enabled, the caching backend gradually serializes query responses for the cache, comparing against the configured `[caching]max_value_mb` value as it goes. This can can help prevent Grafana from running out of memory while attempting to cache very large query responses. | |

| `lokiMetricDataplane` | Changes metric responses from Loki to be compliant with the dataplane specification. | Yes |
| `dataplaneFrontendFallback` | Support dataplane contract field name change for transformations and field name matchers where the name is different | Yes |
| `alertingNotificationsPoliciesMatchingInstances` | Enables the preview of matching instances for notification policies | Yes |
| `useCachingService` | When turned on, the new query and resource caching implementation using a wire service inject will be used in place of the previous middleware implementation | |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| `useCachingService` | When turned on, the new query and resource caching implementation using a wire service inject will be used in place of the previous middleware implementation | |
| `useCachingService` | When active, the new query and resource caching implementation using a wire service inject replaces the previous middleware implementation. | |

| `gcomOnlyExternalOrgRoleSync` | Prohibits a user from changing organization roles synced with Grafana Cloud auth provider | |
| `prometheusMetricEncyclopedia` | Adds the metrics explorer component to the Prometheus query builder as an option in metric select | Yes |
| `prometheusResourceBrowserCache` | Displays browser caching options in Prometheus data source configuration | Yes |
| `prometheusDataplane` | Changes responses to from Prometheus to be compliant with the dataplane specification. In particular it sets the numeric Field.Name from 'Value' to the value of the `__name__` label when present. | Yes |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| `prometheusDataplane` | Changes responses to from Prometheus to be compliant with the dataplane specification. In particular it sets the numeric Field.Name from 'Value' to the value of the `__name__` label when present. | Yes |
| `prometheusDataplane` | Changes responses to from Prometheus to be compliant with the dataplane specification. In particular, when this feature toggle is active, the numeric `Field.Name` is set from 'Value' to the value of the `__name__` label. | Yes |

Copy link
Contributor

@knylander-grafana knylander-grafana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving the doc change with some minor suggestions. Thank you for updating the feature toggle description and transferring these suggested changes to the registry.go file so they can be included when the page is next autogenerated.

@mmandrus mmandrus enabled auto-merge (squash) October 12, 2023 15:29
@mmandrus mmandrus merged commit 94ce875 into main Oct 12, 2023
19 checks passed
@mmandrus mmandrus deleted the mmandrus/gradually-marshal-cache-results branch October 12, 2023 16:56
@zerok zerok modified the milestones: 10.2.x, 10.2.0 Oct 23, 2023
darrenjaneczek pushed a commit that referenced this pull request Oct 23, 2023
…ization (#76145)

* add feature toggle for smart cache serialization

* re-add toggle after merge conflict

* switch feature toggle stage to experimental

* incorporate PR feedback
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants