Skip to content

Bernoulli trial behavior in Tweek #707

@Yshayy

Description

@Yshayy

Currently, gradual release behaves in the following way - for 0.1 ( 10% ) value split the population to two groups

A. 1-10 - TRUE
B. 11-100 - FALSE

Tweek is currently stop evaluating rules after this rule since every group has a value, even group B.
It might be more correct to translate group B value to NONE, which means no match was found and evaluation will continue to the next rule, from semantic point of view, it means we no longer split the population to two groups, but rather add a condition which is like "user is in x% percentile".
After this change, gradual release behaves in a bit "greedier" way.
It can be relevant if we want the users to have more opportunities to get 'true' value.
It can also be relevant for time based releases:

For Key A:
1. CurrentDate > 01/01/2018 - open feature to 10% of users.
2. CurrentDate > 08/01/2018 - open feature to 30% of users.

Assuming theses rules share the same salt, this behavior will behave correct after this change, currently, we will need to add additional constraint for the first rule (date < 08/01/2018) for that to work.

Suggested change:

  1. For rules with gradual release, change Bernoulli trial behavior to use TRUE-NONE instead of TRUE-FALSE, allowing the system to be 'greedy' and move to the next rule.
  2. When creating a new bool key, set default value to FALSE, to avoid null behavior of key when using gradual release.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions