id | slug | title | description | date | tags | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
kibCloudExperimentsPlugin |
/kibana-dev-docs/key-concepts/cloud-experiments-plugin |
Cloud Experiments service |
The Cloud Experiments Service provides the necessary APIs to implement A/B testing scenarios, fetching the variations in configuration and reporting back metrics to track conversion rates of the experiments. |
2022-09-07 |
|
The Cloud Experiments Service provides the necessary APIs to implement A/B testing scenarios, fetching the variations in configuration and reporting back metrics to track conversion rates of the experiments.
The cloudExperiments
plugin is disabled by default and only enabled on Elastic Cloud deployments.
If you are developing a feature that needs to use a feature flag, or you are implementing an A/B-testing scenario, this is how you should fetch the value of your feature flags (for either server and browser side code):
First, you should declare the optional dependency on this plugin. Do not list it in your requiredPlugins
, as this plugin is disabled by default and only enabled in Cloud deployments. Adding it to your requiredPlugins
will cause Kibana to refuse to start by default.
// plugin/kibana.json
{
"id": "myPlugin",
"optionalPlugins": ["cloudExperiments"]
}
Please, be aware that your plugin will run even when the cloudExperiment
plugin is disabled. Make sure to declare it as an optional dependency in your plugin's TypeScript contract to remind you that it might not always be available.
First, make sure that your feature flag is listed in FEATURE_FLAG_NAMES
.
Then, you can fetch the value of your feature flag by using the API cloudExperiments.getVariation
as follows:
import type { CoreSetup, CoreStart, Plugin } from '@kbn/core/(public|server)';
import type {
CloudExperimentsPluginSetup,
CloudExperimentsPluginStart
} from '@kbn/cloud-experiments-plugin/common';
interface SetupDeps {
cloudExperiments?: CloudExperimentsPluginSetup;
}
interface StartDeps {
cloudExperiments?: CloudExperimentsPluginStart;
}
export class MyPlugin implements Plugin<void, void, SetupDeps, StartDeps> {
public setup(core: CoreSetup, deps: SetupDeps) {
this.doSomethingBasedOnFeatureFlag(deps.cloudExperiments);
}
public start(core: CoreStart, deps: StartDeps) {
this.doSomethingBasedOnFeatureFlag(deps.cloudExperiments);
}
private async doSomethingBasedOnFeatureFlag(cloudExperiments?: CloudExperimentsPluginStart) {
let myConfig = 'default config';
if (cloudExperiments) {
myConfig = await cloudExperiments.getVariation(
'my-plugin.my-feature-flag', // The key 'my-plugin.my-feature-flag' should exist in FEATURE_FLAG_NAMES
'default config'
);
}
// do something with the final value of myConfig...
}
}
Since the getVariation
API returns a promise, when using it in a React component, you may want to use the hook useEffect
.
import React, { useEffect, useState } from 'react';
import type {
CloudExperimentsFeatureFlagNames,
CloudExperimentsPluginStart
} from '@kbn/cloud-experiments-plugin/common';
interface Props {
cloudExperiments?: CloudExperimentsPluginStart;
}
const useVariation = <Data>(
cloudExperiments: CloudExperimentsPluginStart | undefined,
featureFlagName: CloudExperimentsFeatureFlagNames,
defaultValue: Data,
setter: (value: Data) => void
) => {
useEffect(() => {
(async function loadVariation() {
const variationUrl = await cloudExperiments?.getVariation(featureFlagName, defaultValue);
if (variationUrl) {
setter(variationUrl);
}
})();
}, [cloudExperiments, featureFlagName, defaultValue, setter]);
};
export const MyReactComponent: React.FC<Props> = ({ cloudExperiments }: Props) => {
const [myConfig, setMyConfig] = useState('default config');
useVariation(
cloudExperiments,
'my-plugin.my-feature-flag', // The key 'my-plugin.my-feature-flag' should exist in FEATURE_FLAG_NAMES
'default config',
setMyConfig
);
// use myConfig in the component...
}
Experiments require feedback to analyze which variation to the feature flag is the most successful. For this reason, we need to report some metrics defined in the success criteria of the experiment (check back with your PM if they are unclear).
Our A/B testing provider allows some high-level analysis of the experiment based on the metrics. It also has some limitations about how it handles some type of metrics like number of objects or size of indices. For this reason, you might want to consider shipping the metrics via our usual telemetry channels (core.analytics
for event-based metrics, or ).
However, if our A/B testing provider's analysis tool is good enough for your use case, you can use the api reportMetric
as follows.
First, make sure to add the metric name in METRIC_NAMES
. Then you can use it like below:
import type { CoreStart, Plugin } from '@kbn/core/(public|server)';
import type {
CloudExperimentsPluginSetup,
CloudExperimentsPluginStart
} from '@kbn/cloud-experiments-plugin/common';
interface SetupDeps {
cloudExperiments?: CloudExperimentsPluginSetup;
}
interface StartDeps {
cloudExperiments?: CloudExperimentsPluginStart;
}
export class MyPlugin implements Plugin<void, void, SetupDeps, StartDeps> {
public start(core: CoreStart, deps: StartDeps) {
// whenever we need to report any metrics:
// the user performed some action,
// or a metric hit a threshold we want to communicate about
deps.cloudExperiments?.reportMetric({
name: 'Something happened', // The key 'Something happened' should exist in METRIC_NAMES
value: 22, // (optional) in case the metric requires a numeric metric
meta: { // Optional metadata.
hadSomething: true,
userType: 'type 1',
otherNumericField: 1,
}
})
}
}
To test your code locally when developing the A/B scenarios, this plugin accepts a custom config to skip the A/B provider calls and return the values. Use the following kibana.dev.yml
configuration as an example:
xpack.cloud_integrations.experiments.enabled: true
xpack.cloud_integrations.experiments.flag_overrides:
"my-plugin.my-feature-flag": "my custom value"
The user is automatically identified during the setup
phase. It currently uses the ESS deployment ID, meaning all users accessing the same deployment will get the same values for the getVariation
requests unless the A/B provider is explicitly configured to randomize it.
If you are curious of the data provided to the identify
call, you can see that in the cloud
plugin.
See the kibana contributing guide for instructions setting up your development environment.