feat: benchmarking option in CLI

[polok]

What's done

Introduced an option to benchmark feature/variation/variable evaluations from CLI as JS SDK did (more info here)

API

Read more in README (in diff).

To run benchmarking for a feature:

$  FeaturevisorTestRunner benchmark \
  --environment staging \
  --feature feature_key \
  --context '{"user_id":"123"}' \
  -n 100

Output will be similar to this:

Running benchmark for feature mob_eos_ios...
Building datafile containing all features for staging...
Datafile build duration: 0.197859334s
...SDK initialized
Against context: ["user_id": FeaturevisorTypes.AttributeValue.string("123")]
Evaluating flag 100 times...
Evaluated value : Optional(true)
Total duration  : 0.9339580000000001ms
Average duration: 0.00933958ms

[fahad19]

great stuff as always, @polok!

a note about future (not this PR), given the test runner in Swift is going a bit beyond just running tests, we may want to rethink how we wish to name it more generic instead of FeaturevisorTestRunner.

[polok]

@fahad19 Yessss, I thought about the same when introduced new command benchmark and putting existing under command test. It's more like featurevisor-swift CLI now (maybe FeaturevisorSwift or FeaturevisorSwiftCLI). What do you think?

[fahad19]

@polok: I like your suggestion very much!

I expect more SDKs to be released this year, and having language specific command names for each of them would be very handy, like:

• $ featurevisor-swift
• $ featurevisor-go

I also understand that not all languages would want to expose their CLIs as an independent binary (like Go might).

so it's perfectly fine if they are run via task runners like $ gradle and in the case of Swift $ swift.

it's the subcommands like benchmark, test, evaluate along with their options/flags that I would focus more on for maintaining feature parity: https://featurevisor.com/docs/cli/