Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose connection state to client consumers #29

Open
aalexandrov opened this issue Dec 13, 2022 · 3 comments
Open

Expose connection state to client consumers #29

aalexandrov opened this issue Dec 13, 2022 · 3 comments

Comments

@aalexandrov
Copy link

Is your feature request related to a problem? Please describe.

At Materialize, we plan to use LaunchDarkly in continuously running processes and would like to have an internal metric indicating a state where the client instance temporary could not communicate with the LaunchDarkly service.

Describe the solution you'd like

Some method on the client instance that returns a bool or an enum indicates whether the client is connected and the connection is alive.

Describe alternatives you've considered
Periodically polling LaunchDarkly API in a separate process / Tokio task.

Additional context

As a cloud service that is meant to be used in critical business applications, it is important to have good visibility for all our external dependencies.

@keelerm84
Copy link
Member

Thank you for filing this feature request. We have an internal ticket (sc-142483) tracking support for this feature. We will update this issue when the feature has been released.

aalexandrov pushed a commit to MaterializeInc/rust-server-sdk that referenced this issue Dec 15, 2022
* refactor HTTP client into a lib

* Working eventsource client

* Scaffold up Client, StreamingUpdateProcessor, FeatureStore

* fixup! Working eventsource client

* fixup! Scaffold up Client, StreamingUpdateProcessor, FeatureStore

* split up into multiple files

* Skeleton implementation of streaming eval - returns killswitch as flag value

* allow setting stream url in hack script

* fix default stream url

* TODO WIP handle partial lines

* flesh out buffering decoder (but wrong)

* dedupe common parsing logic between two decoder impls

* simplify with try_ready macro

* poll in a loop so we keep processing data from the stream

* Implement partial chunk buffering

* remove now-unused chunkwise decoder

* despammify debug logging a bit

* more structured error handling

* more error handling improvements

* add some tests and firm up error handling a bit

* simplify incomplete line handling, and fix eof handling

* use mem::replace to avoid copying event

* move hack script to examples/ so it can use dev-dependencies

* Clarify stream code, distinguish request errors from stream errors

* Use log crate for logging, and clean up log code

* cleanup todos

* example script: pass flags to watch as env vars

* flag eval: implement off-variation and fallthrough variation

* flag eval: implement provided default

* minor cleanup

* implement eval reasons to aid debugging

* flag eval: implement user targeting

* return our own FlagValue enum instead of exposing serde_json::Value

* define FeatureFlag struct

* strongly type more things

* parse variations into our FlagValue enum

* make creating users slightly easier

* tests for flag evaluation so far

* Implement basic rule evaluation

* support negated clauses

* add TODOs for missing implementation

* extract eventsource into a separate package

* add more TODOs

* start implementing event sending (no batching yet)

* clone() wars

* reduce verbosity by default

* make event-processor robust to tokio runtime not started yet

* add todo note

* implement a working demo with a non-tokio main thread

* implement string flags

* support null off_variation

* event sending shouldn't panic if value is None

* parse rollout (but not implemented yet)

* parse some flag rule operations (not implemented yet though)

* handle malformed 'fallthrough: {}'

* Handle patches with segment data (don't actually support segments yet though)

* Handle deletes

* Support all_flags (TODO don't send events)

* implement float and int

* clean up unimpl logging for segments

* restructure eval Detail so we can return variationIndex

* Allow user keys to be null

* missing user key should short-circuit flag eval and return default

* Log error if sending event receives non-successful response

* implement some string ops

* Implement Op::Matches

* serialize reasons and errors as IH expects

* support clauses matching against key

* implement numeric attributes and operators

* builder API for user, support name and email

* Implement all non-custom attributes

* Restore event attribute names after attribute refactoring

* Implement Deserialize for User, mostly for restwrapper

* implement config.inline_users_in_events

* don't output a bunch of tedious nulls

* Fix a couple of warnings

* implement Before and After operators

* Add circle config

* use published eventsource-client crate instead of local checkout

* Allow attributes to be null

* implement some clippy suggestions

* Implement more clippy suggestions

* and more clippy

* rejigger default handling so we can report value and default correctly in events

* got ahead of ourselves a bit

* fix clippy suggestions

* change how user inlining is serialized

* include variation index in feature event

* support i64 as attribute value

* add User.attribute mutator, and extra fun to ncurses demo

* Fake summary events (batch size = 1)

* refactor event processor to allow testing it

* use nicer json literals

* Emit reason in feature events

* Refactor client to allow (some) testing

* Allow mocking UpdateProcessor, and add overly broad test that does so

* jiggle around eval so we send events for null user keys

* Add a detail-free evaluate method that omits rather than emits reasons

* implement JSON conversions for FlagValue for restwrapper

* Remove redundant user key check

* partial impl of prereqs (TODO: events, some bugs)

* make eventsink sink events rather than JSON

* batch events, and replace hacky tokio sender with a background thread

* get rid of simplelog it is not simple

* implement flush for events

* Use batcher to generate real summary events, and transform them into the right format

* Implement rollouts

* support int attributes as well as floats, so can disallow bucketing by float

* implement segments and segmentMatch operator

* Partially implement semver operations

* fix formatting and move format check to after build

* fix clippy warnings

* fix more clippy warnings

* Remove some unnecessary tokio cruft from demo

* Make it easier to start client in a background thread

* support configuring LD urls in demo

* Declunk

* Ensure client is started exactly once

* clippy

* Rationalise client::Error

* clippy

* implement JSON and Int variations

* we clone str_variation default anyway, pass it as owned

* add non-detail variation methods for all types

* don't convert ints into floats unnecessarily

* remove boilerplate test

* dedupe users per batch

* add user agent

* forgot to add build.rs

* cargo fmt

* add stripped down readme

* add changelog contributing and license

* add .github issue and PR templates

* Bound http crate version to avoid HttpTryFrom compile errors

* dedupe user-agent construction using lazy_static

* Remove now-unused imports

* More generic user builder (#3)

* implement LD-specific semver extended semantics

* if prereq is off, return off, regardless of eval result

* ok clippy

* add identify

* implement track

* Disallow null user keys

* [ch97491] Fix clippy warnings about prefix stripping

The new version of clippy is more restrictive causing CI to fail.

* Removed the guides link

* Bump eventsource-client to 0.5.0 for decoder fixes

* okay clippy

* Return rule ID in reasons (#5)

This adds support for passing through the matched rule id

* [ch102632] use library versions of VariationIndex, Detail, Reason, Error

* [ch102632] use library versions of Flag, Segment, Rule, and friends

* [ch102632] placate clippy

* [ch102632] formatting

* Shake out implications of user.key() returning &str (launchdarkly#7)

Changing user.key() to return &str created a few compile errors in the SDK - this fixes them.

* Don't send feature event unless we should (launchdarkly#8)

Previously the Rust SDK was always sending a full "kind": "feature" event for every evaluation, in addition to the summary events. This disables those events, unless either the flag-level trackEvents property was set, or the evaluation hit a rule or fallthrough that had their respective trackEvents properties set.

I also implemented the very-special logic whereby we sometimes send reasons even if the caller didn't ask for them, and refactored the summary event logic to make it clearer that it only summarises feature events.

* don't test with int too large to represent

* example - fix unused Result warning after crate update

* Cleanup TODOs (launchdarkly#9)

Ditches a couple of obsolete TODOs and ensures all others (except a couple in the Circle config) are associated with CH tickets.

* Clean up exported surface area (launchdarkly#10)

This cleans up the SDK crate exports to just what should be needed to actually configure a client, create users and evaluate flags. It also finally brings the crate name in line with our naming conventions.

* Update tests for latest evaluation changes (launchdarkly#13)

* Simply URL trimming functionality (launchdarkly#14)

* Separate out new_unknown_event from new_feature_request (launchdarkly#15)

* Add contextKind designation to events (launchdarkly#16)

* Make custom events own struct

* Add support for alias events (launchdarkly#17)

* Port to newer async libraries (and compatibility with eventsource-client 0.6) (launchdarkly#12)

* Start implementing builders similar to the other SDKs. (launchdarkly#19)

* Rename update processor to data source

* Introduce data store builder pattern

* Add tests for data store (launchdarkly#21)

* sc-131624 Because everything inside client is Send+Sync safe now, the… (launchdarkly#27)

* Rlamb/sc 108602/implement client not ready (launchdarkly#28)

* sc-131893 Refactor starting with runtime. (launchdarkly#30)

* Add prereq event support (launchdarkly#26)

* Separate out error types (launchdarkly#25)

* Add version check for flag updates (launchdarkly#29)

* Remove test duplication and expand coverage (launchdarkly#33)

* SC-131531: Add code coverage & refactor config.yaml (launchdarkly#31)

* Update config.yml to include rust code coverage (via coverage.sh)

* Refactor config.yml

* Detail.value might be none now (launchdarkly#35)

With the introduction of prereq events, Detail.value might be None when
evaluating events. This is because we do not have a default value to
provide, and thus the evaluation process might yield a None result.

* Improving all_flags_detail consistency (launchdarkly#38)

* Update evaluate language (launchdarkly#37)

* Move inline_users_in_events into event processor (launchdarkly#39)

* Ensure client is always send + sync (#40)

* Support offline mode (launchdarkly#41)

* Add support for debug events (launchdarkly#42)

* Add version_string() function to fetch SDK version (launchdarkly#36)

* Fix failing tests (launchdarkly#44)

* Improve event processor spec alignment (launchdarkly#45)

The add and flush methods are not supposed to block or return success or
failure condition. However, if an event exceeds the capacity of the
event queue, we are supposed to notify the user this has occurred one
time.

Originally we were triggering flushes based off a batch size and also
some max batch deadline. The spec requires neither of these, so I have
removed them. This addresses the issue of requiring additional
configuration and takes us one step closer to alignment in this
processor.

I also did away with the separate flush channel. The spec requires only
a single inbox channel which includes messages for both events and flush
requests.

* Update eval crate name and path (launchdarkly#50)

* Favor error enum over String (launchdarkly#48)

* Add secure_mode_hash support (launchdarkly#46)

* Add support for user_keys_flush_interval (launchdarkly#47)

* Improve json f64 serialization (launchdarkly#49)

The default float serialization process in serde_json is fast but isn't
quite as precise as implementations in other languages. See
[this serde_json PR][1]. By enabling this feature flag, we can bring the
parsing logic inline with other language standards.

[1]: serde-rs/json#536

* Improve event processor compliance, readability, and memory (launchdarkly#51)

This commit addresses several issues:

- Adds support for multiple flush workers as defined by the spec
- Adds basic retry support when sending event payloads, including
  handling of retryable error codes
- Better separates out responsibilities which led to improved testing
- Provides the ability to stop the event processor thread
- Makes the event outbox a fixed size to prevent constant dynamic
  resizing
- Also makes the outbox reusable instead of re-allocating a new vector
  each time.

* Add polling support (launchdarkly#43)

* Remove locking when possible (launchdarkly#52)

We can remove locking from the event processor and the data source as it
doesn't mutate it's own state, and it is fully owned by the client.

The data store both mutates it's own internal state and is shared
between the data source and the client. We cannot remove the locking
functionality, but we can replace the Mutex with a RwLock
implementation.

We opt for using the parking_lot::RwLock instead of the
std::sync::RwLock because the built-in one on Linux suffers from writer
starvation whereas the parking_lot implementation offers a fair locking
mechanism.

* Prevent double EventProcessor::close call from blocking (launchdarkly#56)

* Shutdown data source on Client::close (launchdarkly#55)

* Update usage to reflect latest eval client side change (launchdarkly#59)

* Update various repo templates (launchdarkly#57)

* Add support for releaser (launchdarkly#58)

* Update documentation to prepare for upcoming release (launchdarkly#54)

* Update SDK to use published crates (launchdarkly#60)

* Final fixes for beta release (launchdarkly#61)

* Keyword is too long

* Add SDK contract tests (launchdarkly#63)

* Add creation date to alias and emit index events properly (launchdarkly#65)

* Fix crate repo metadata and circleci badge (launchdarkly#62)

* Remove dependency on OpenSSL (launchdarkly#64)

* Updating changelog and version for 1.0.0-beta.2 release

* Fix README in published package (launchdarkly#66)

crates.io only shows a README if it's at the root of the project that is
being published. To fix this, and ensure that GitHub renders the same
README, we symlink the package README to the root of the repository.

* [sc-140521] Add the ability to re-use a connector object between clients. (launchdarkly#68)

* Remove reliance on spectral (launchdarkly#69)

* Enable HTTPS support for reqwest (launchdarkly#70)

* Handle negative variation indexes (launchdarkly#71)

The SDK test harness verifies that the SDK can handle invalid flag data
(such as negative variation indexes). The evaluation crate was updated
to deal with this situation. This change brings the SDK inline with the
new evaluation types.

* Support private attributes (launchdarkly#67)

* Bump versions for several packages (launchdarkly#73)

* Add support for adding tags to all LD requests (launchdarkly#74)

* Update eventsource-client dependency (launchdarkly#76)

* Update data_source to use version 0.9.0 of eventsource-client

* Update Makefile dependencies

* Update eventsource error handling to take into account EOF

* Refactor the StreamingDataSource builder to put common building code into single function

* Update from Rust 2018 -> Rust 2021 Edition (launchdarkly#80)

cargo fix --edition had nothing to say.

* Bump eventsource-client to 0.10.0 (launchdarkly#81)

Pulls in 301 & 307 redirect handling from upstream, as well as fixed handling of Last-Event-ID.

* Add support for persistent data stores (launchdarkly#72) (launchdarkly#77)

This work introduces the required structure to support persistent data
stores like Redis. It includes the PersistentDataStoreWrapper and the
associated caching methods.

Along with these changes, the file structure was reshaped to more
clearly separate out store related blocks of code. Work was also done to
remove the patch and delete methods, favoring instead a single upsert
method.

Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>

* Pin beta versions more strictly (launchdarkly#84)

* Preparing 1.0.0-beta.3 (launchdarkly#85)

Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>

* Add X-LaunchDarkly-PayloadID and X-LaunchDarkly-Event-Schema headers (launchdarkly#88)

Adds missing headers to event sending code.

The payload ID allows event recorder to dedup events if the SDK sends them multiple times (because of retries)
or if some other upstream service duplicates them.

The current event-schema version is 3.

* Implement service-endpoint capability for contract tests (launchdarkly#90)

* Implement service-endpoint capability for contract tests

* Ensure event summary information is reset thoroughly (#92)

* Restrict tag value length (#93)

* master -> main (#94)

* Update to cimg/rust:1.60.0 and remove nightly toolchain (#95)

* Update to cimg/rust:1.60.0

This update allows us to remove the nightly toolchain that was necessary for generating code coverage.

* ci: use large executor class

Medium+ seems to be running into memory exhaustion when linking in the test step. Reducing the number of parallel builds in the test
using -j1 did not help.

* Prevent sending shutdown broadcast when in offline mode (#97)

* Add polling capability to test harness (#99)

* [sc-169541] Fix temporary/permanent polling failure and allow for retry of initial poll. (#100)

* Bump eventsource-client to 0.11.0 (#107)

This includes support for the retry jitter and backoff reset if the
connect remains stable.

* fix script permissions

* Merge v1 into main (#117)

* Remove alias events and inline users option (launchdarkly#75)

* Add support for persistent data stores (launchdarkly#72)

This work introduces the required structure to support persistent data
stores like Redis. It includes the PersistentDataStoreWrapper and the
associated caching methods.

Along with these changes, the file structure was reshaped to more
clearly separate out store related blocks of code. Work was also done to
remove the patch and delete methods, favoring instead a single upsert
method.

Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>

* Update to Rust 2021 Edition

* Updating SDK with new context evaluation logic (launchdarkly#78)

* Update contract tests to handle contextBuild/contextConvert commands (launchdarkly#83)

* Add contextBuild command to contract-tests

* Use server-sdk's symbols instead of directly importing evaluation crate in contract-test

* Remove passing contract tests from testharness-suppressions.txt

* Update events to support contexts (launchdarkly#82)

* Update test harness suppression file with latest status (launchdarkly#86)

* Address retry behavior from contract tests (launchdarkly#87)

* Fix evaluation pathing from merge forward

* Remove literal attribute test suppression (launchdarkly#89)

* Update v1 with latest changes from master (launchdarkly#91)

* Add X-LaunchDarkly-PayloadID and X-LaunchDarkly-Event-Schema headers (launchdarkly#88)

Adds missing headers to event sending code.

The payload ID allows event recorder to dedup events if the SDK sends them multiple times (because of retries)
or if some other upstream service duplicates them.

The current event-schema version is 3.

* Implement service-endpoint capability for contract tests (launchdarkly#90)

* Rename transient back to anonymous (#96)

* Rename transient back to anonymous, using updated evaluation crate APIs.

* Fix event summary test from main merge (#98)

* Allow using hyper for polling and event sending. (#102)

Co-authored-by: Matthew M. Keeler <mkeeler@launchdarkly.com>

* Fix support for HTTPS with hyper (#104)

* Change streaming max delay to 30s (#103)

According to the [streaming spec][1], the maximum retry delay should be
set to 30 secondary.

[1]: https://launchdarkly.atlassian.net/wiki/spaces/ENG/pages/521668284/SDK+streaming+specification

* Add 1 second sleep between event sender retries (#105)

In accordance with the [event processor spec][1], we need to sleep once
after an initial failed attempt.

[1]: https://launchdarkly.atlassian.net/wiki/spaces/ENG/pages/356745688/SDK+event+processor+design

* Accept a wider range of private attribute values (#108)

Originally we were only accepting private attribute values that were
`String`s. This is limiting, especially if you want to pre-calculate the
`Reference` instances for better performance. The
`EventProcessorBuilder` now accepts any `HashSet<Into<Reference>>`.

Additionally, I modified it so that these values are converted to
`Reference` instances during the time of construction instead of during
event generation. Construction is way less frequent, so this should
provide some amount of speed improvement as well.

* Remove secondary support & update contract tests (#109)

* tests: update some more raw JSON strings to use json! macro

* contract tests: remove secondary handling

* events: remove secondary from event serialization tests; add new test for local private attribute

* Upgrade various 3rd party libs & set MSRV (#111)

* Update dependency moka from 7.1 to 9.6

* Upgrade lru from 7.2 to 8.1

* Upgrade uuid from alpha 1.0 release to 1.2.2

* Upgrade hyper-rustls from 22.1 to 23.1

* Update env_logger from 0.9 to 0.10

* contract tests: update eventsource-client from 0.10 to 0.11

* contract tests: upgrade env_logger from 0.9 to 0.10

* contract tests: upgrade actix from 0.12 to 0.13

* contract tests: bump actix-web off beta version to 4.2.1

* Update CircleCI musl executor to 1.60.0, and bump MSRV to 1.60.0

Since >= 1 of our dependencies do this now (env_logger), we should declare a
minimum supported rust version.

* hyper-rustls: enable http2 explicitely

* Update to integrate latest eval crate changes (#112)

* Integrate eval crate changes

* Fix a bunch of linter warnings (#113)

* Cleanup a batch of TODOs (#114)

* Removed todo about diagnostic event config, as it is now obsolete
* Removed multiple todos about ergonomics of detail struct (still needs to be fixed but exists as ticket)
* Removed todo about docs on ConfigBuilder, and updated those docs
* Removed todo about backoff/jitter on MockDataSourceBuilder
* Removed obsolete event tests
* Removed todo on InMemoryDataStore
* Re-Exported the 'Kind' type from the eval crate

* Bump eval crate to public 1.0.0 release (#115)

* Remove todo in streaming data source (#116)

State machine behavior to be audited at a later date.

* Remove beta warning from README (#118)

Co-authored-by: Matthew M. Keeler <mkeeler@launchdarkly.com>
Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>

Co-authored-by: Sam Stokes <sstokes@launchdarkly.com>
Co-authored-by: Harpo Roeder <hroeder@launchdarkly.com>
Co-authored-by: Ben Woskow <bwoskow@launchdarkly.com>
Co-authored-by: Ben Woskow <48036130+bwoskow-ld@users.noreply.github.com>
Co-authored-by: Ezra Stevens <estevens@launchdarkly.com>
Co-authored-by: Matthew M. Keeler <keelerm84@gmail.com>
Co-authored-by: Matthew Keeler <mkeeler@launchdarkly.com>
Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>
Co-authored-by: Casey Waldren <cwaldren@launchdarkly.com>
Co-authored-by: Eli Bishop <eli@launchdarkly.com>
@benesch
Copy link

benesch commented Dec 17, 2022

Thanks! @aalexandrov, correct me if I'm wrong, but I think even better than a bool would be the time of the last successful synchronization, or the last successful connection heartbeat—whatever makes sense given the design of the internals of the library.

@aalexandrov
Copy link
Author

@benesch yes a timestamp would be better.

aalexandrov pushed a commit to aalexandrov/rust-server-sdk that referenced this issue Dec 22, 2022
* refactor HTTP client into a lib

* Working eventsource client

* Scaffold up Client, StreamingUpdateProcessor, FeatureStore

* fixup! Working eventsource client

* fixup! Scaffold up Client, StreamingUpdateProcessor, FeatureStore

* split up into multiple files

* Skeleton implementation of streaming eval - returns killswitch as flag value

* allow setting stream url in hack script

* fix default stream url

* TODO WIP handle partial lines

* flesh out buffering decoder (but wrong)

* dedupe common parsing logic between two decoder impls

* simplify with try_ready macro

* poll in a loop so we keep processing data from the stream

* Implement partial chunk buffering

* remove now-unused chunkwise decoder

* despammify debug logging a bit

* more structured error handling

* more error handling improvements

* add some tests and firm up error handling a bit

* simplify incomplete line handling, and fix eof handling

* use mem::replace to avoid copying event

* move hack script to examples/ so it can use dev-dependencies

* Clarify stream code, distinguish request errors from stream errors

* Use log crate for logging, and clean up log code

* cleanup todos

* example script: pass flags to watch as env vars

* flag eval: implement off-variation and fallthrough variation

* flag eval: implement provided default

* minor cleanup

* implement eval reasons to aid debugging

* flag eval: implement user targeting

* return our own FlagValue enum instead of exposing serde_json::Value

* define FeatureFlag struct

* strongly type more things

* parse variations into our FlagValue enum

* make creating users slightly easier

* tests for flag evaluation so far

* Implement basic rule evaluation

* support negated clauses

* add TODOs for missing implementation

* extract eventsource into a separate package

* add more TODOs

* start implementing event sending (no batching yet)

* clone() wars

* reduce verbosity by default

* make event-processor robust to tokio runtime not started yet

* add todo note

* implement a working demo with a non-tokio main thread

* implement string flags

* support null off_variation

* event sending shouldn't panic if value is None

* parse rollout (but not implemented yet)

* parse some flag rule operations (not implemented yet though)

* handle malformed 'fallthrough: {}'

* Handle patches with segment data (don't actually support segments yet though)

* Handle deletes

* Support all_flags (TODO don't send events)

* implement float and int

* clean up unimpl logging for segments

* restructure eval Detail so we can return variationIndex

* Allow user keys to be null

* missing user key should short-circuit flag eval and return default

* Log error if sending event receives non-successful response

* implement some string ops

* Implement Op::Matches

* serialize reasons and errors as IH expects

* support clauses matching against key

* implement numeric attributes and operators

* builder API for user, support name and email

* Implement all non-custom attributes

* Restore event attribute names after attribute refactoring

* Implement Deserialize for User, mostly for restwrapper

* implement config.inline_users_in_events

* don't output a bunch of tedious nulls

* Fix a couple of warnings

* implement Before and After operators

* Add circle config

* use published eventsource-client crate instead of local checkout

* Allow attributes to be null

* implement some clippy suggestions

* Implement more clippy suggestions

* and more clippy

* rejigger default handling so we can report value and default correctly in events

* got ahead of ourselves a bit

* fix clippy suggestions

* change how user inlining is serialized

* include variation index in feature event

* support i64 as attribute value

* add User.attribute mutator, and extra fun to ncurses demo

* Fake summary events (batch size = 1)

* refactor event processor to allow testing it

* use nicer json literals

* Emit reason in feature events

* Refactor client to allow (some) testing

* Allow mocking UpdateProcessor, and add overly broad test that does so

* jiggle around eval so we send events for null user keys

* Add a detail-free evaluate method that omits rather than emits reasons

* implement JSON conversions for FlagValue for restwrapper

* Remove redundant user key check

* partial impl of prereqs (TODO: events, some bugs)

* make eventsink sink events rather than JSON

* batch events, and replace hacky tokio sender with a background thread

* get rid of simplelog it is not simple

* implement flush for events

* Use batcher to generate real summary events, and transform them into the right format

* Implement rollouts

* support int attributes as well as floats, so can disallow bucketing by float

* implement segments and segmentMatch operator

* Partially implement semver operations

* fix formatting and move format check to after build

* fix clippy warnings

* fix more clippy warnings

* Remove some unnecessary tokio cruft from demo

* Make it easier to start client in a background thread

* support configuring LD urls in demo

* Declunk

* Ensure client is started exactly once

* clippy

* Rationalise client::Error

* clippy

* implement JSON and Int variations

* we clone str_variation default anyway, pass it as owned

* add non-detail variation methods for all types

* don't convert ints into floats unnecessarily

* remove boilerplate test

* dedupe users per batch

* add user agent

* forgot to add build.rs

* cargo fmt

* add stripped down readme

* add changelog contributing and license

* add .github issue and PR templates

* Bound http crate version to avoid HttpTryFrom compile errors

* dedupe user-agent construction using lazy_static

* Remove now-unused imports

* More generic user builder (launchdarkly#3)

* implement LD-specific semver extended semantics

* if prereq is off, return off, regardless of eval result

* ok clippy

* add identify

* implement track

* Disallow null user keys

* [ch97491] Fix clippy warnings about prefix stripping

The new version of clippy is more restrictive causing CI to fail.

* Removed the guides link

* Bump eventsource-client to 0.5.0 for decoder fixes

* okay clippy

* Return rule ID in reasons (launchdarkly#5)

This adds support for passing through the matched rule id

* [ch102632] use library versions of VariationIndex, Detail, Reason, Error

* [ch102632] use library versions of Flag, Segment, Rule, and friends

* [ch102632] placate clippy

* [ch102632] formatting

* Shake out implications of user.key() returning &str (launchdarkly#7)

Changing user.key() to return &str created a few compile errors in the SDK - this fixes them.

* Don't send feature event unless we should (launchdarkly#8)

Previously the Rust SDK was always sending a full "kind": "feature" event for every evaluation, in addition to the summary events. This disables those events, unless either the flag-level trackEvents property was set, or the evaluation hit a rule or fallthrough that had their respective trackEvents properties set.

I also implemented the very-special logic whereby we sometimes send reasons even if the caller didn't ask for them, and refactored the summary event logic to make it clearer that it only summarises feature events.

* don't test with int too large to represent

* example - fix unused Result warning after crate update

* Cleanup TODOs (launchdarkly#9)

Ditches a couple of obsolete TODOs and ensures all others (except a couple in the Circle config) are associated with CH tickets.

* Clean up exported surface area (launchdarkly#10)

This cleans up the SDK crate exports to just what should be needed to actually configure a client, create users and evaluate flags. It also finally brings the crate name in line with our naming conventions.

* Update tests for latest evaluation changes (launchdarkly#13)

* Simply URL trimming functionality (launchdarkly#14)

* Separate out new_unknown_event from new_feature_request (launchdarkly#15)

* Add contextKind designation to events (launchdarkly#16)

* Make custom events own struct

* Add support for alias events (launchdarkly#17)

* Port to newer async libraries (and compatibility with eventsource-client 0.6) (launchdarkly#12)

* Start implementing builders similar to the other SDKs. (launchdarkly#19)

* Rename update processor to data source

* Introduce data store builder pattern

* Add tests for data store (launchdarkly#21)

* sc-131624 Because everything inside client is Send+Sync safe now, the… (launchdarkly#27)

* Rlamb/sc 108602/implement client not ready (launchdarkly#28)

* sc-131893 Refactor starting with runtime. (launchdarkly#30)

* Add prereq event support (launchdarkly#26)

* Separate out error types (launchdarkly#25)

* Add version check for flag updates (launchdarkly#29)

* Remove test duplication and expand coverage (launchdarkly#33)

* SC-131531: Add code coverage & refactor config.yaml (launchdarkly#31)

* Update config.yml to include rust code coverage (via coverage.sh)

* Refactor config.yml

* Detail.value might be none now (launchdarkly#35)

With the introduction of prereq events, Detail.value might be None when
evaluating events. This is because we do not have a default value to
provide, and thus the evaluation process might yield a None result.

* Improving all_flags_detail consistency (launchdarkly#38)

* Update evaluate language (launchdarkly#37)

* Move inline_users_in_events into event processor (launchdarkly#39)

* Ensure client is always send + sync (#40)

* Support offline mode (launchdarkly#41)

* Add support for debug events (launchdarkly#42)

* Add version_string() function to fetch SDK version (launchdarkly#36)

* Fix failing tests (launchdarkly#44)

* Improve event processor spec alignment (launchdarkly#45)

The add and flush methods are not supposed to block or return success or
failure condition. However, if an event exceeds the capacity of the
event queue, we are supposed to notify the user this has occurred one
time.

Originally we were triggering flushes based off a batch size and also
some max batch deadline. The spec requires neither of these, so I have
removed them. This addresses the issue of requiring additional
configuration and takes us one step closer to alignment in this
processor.

I also did away with the separate flush channel. The spec requires only
a single inbox channel which includes messages for both events and flush
requests.

* Update eval crate name and path (launchdarkly#50)

* Favor error enum over String (launchdarkly#48)

* Add secure_mode_hash support (launchdarkly#46)

* Add support for user_keys_flush_interval (launchdarkly#47)

* Improve json f64 serialization (launchdarkly#49)

The default float serialization process in serde_json is fast but isn't
quite as precise as implementations in other languages. See
[this serde_json PR][1]. By enabling this feature flag, we can bring the
parsing logic inline with other language standards.

[1]: serde-rs/json#536

* Improve event processor compliance, readability, and memory (launchdarkly#51)

This commit addresses several issues:

- Adds support for multiple flush workers as defined by the spec
- Adds basic retry support when sending event payloads, including
  handling of retryable error codes
- Better separates out responsibilities which led to improved testing
- Provides the ability to stop the event processor thread
- Makes the event outbox a fixed size to prevent constant dynamic
  resizing
- Also makes the outbox reusable instead of re-allocating a new vector
  each time.

* Add polling support (launchdarkly#43)

* Remove locking when possible (launchdarkly#52)

We can remove locking from the event processor and the data source as it
doesn't mutate it's own state, and it is fully owned by the client.

The data store both mutates it's own internal state and is shared
between the data source and the client. We cannot remove the locking
functionality, but we can replace the Mutex with a RwLock
implementation.

We opt for using the parking_lot::RwLock instead of the
std::sync::RwLock because the built-in one on Linux suffers from writer
starvation whereas the parking_lot implementation offers a fair locking
mechanism.

* Prevent double EventProcessor::close call from blocking (launchdarkly#56)

* Shutdown data source on Client::close (launchdarkly#55)

* Update usage to reflect latest eval client side change (launchdarkly#59)

* Update various repo templates (launchdarkly#57)

* Add support for releaser (launchdarkly#58)

* Update documentation to prepare for upcoming release (launchdarkly#54)

* Update SDK to use published crates (launchdarkly#60)

* Final fixes for beta release (launchdarkly#61)

* Keyword is too long

* Add SDK contract tests (launchdarkly#63)

* Add creation date to alias and emit index events properly (launchdarkly#65)

* Fix crate repo metadata and circleci badge (launchdarkly#62)

* Remove dependency on OpenSSL (launchdarkly#64)

* Updating changelog and version for 1.0.0-beta.2 release

* Fix README in published package (launchdarkly#66)

crates.io only shows a README if it's at the root of the project that is
being published. To fix this, and ensure that GitHub renders the same
README, we symlink the package README to the root of the repository.

* [sc-140521] Add the ability to re-use a connector object between clients. (launchdarkly#68)

* Remove reliance on spectral (launchdarkly#69)

* Enable HTTPS support for reqwest (launchdarkly#70)

* Handle negative variation indexes (launchdarkly#71)

The SDK test harness verifies that the SDK can handle invalid flag data
(such as negative variation indexes). The evaluation crate was updated
to deal with this situation. This change brings the SDK inline with the
new evaluation types.

* Support private attributes (launchdarkly#67)

* Bump versions for several packages (launchdarkly#73)

* Add support for adding tags to all LD requests (launchdarkly#74)

* Update eventsource-client dependency (launchdarkly#76)

* Update data_source to use version 0.9.0 of eventsource-client

* Update Makefile dependencies

* Update eventsource error handling to take into account EOF

* Refactor the StreamingDataSource builder to put common building code into single function

* Update from Rust 2018 -> Rust 2021 Edition (launchdarkly#80)

cargo fix --edition had nothing to say.

* Bump eventsource-client to 0.10.0 (launchdarkly#81)

Pulls in 301 & 307 redirect handling from upstream, as well as fixed handling of Last-Event-ID.

* Add support for persistent data stores (launchdarkly#72) (launchdarkly#77)

This work introduces the required structure to support persistent data
stores like Redis. It includes the PersistentDataStoreWrapper and the
associated caching methods.

Along with these changes, the file structure was reshaped to more
clearly separate out store related blocks of code. Work was also done to
remove the patch and delete methods, favoring instead a single upsert
method.

Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>

* Pin beta versions more strictly (launchdarkly#84)

* Preparing 1.0.0-beta.3 (launchdarkly#85)

Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>

* Add X-LaunchDarkly-PayloadID and X-LaunchDarkly-Event-Schema headers (launchdarkly#88)

Adds missing headers to event sending code.

The payload ID allows event recorder to dedup events if the SDK sends them multiple times (because of retries)
or if some other upstream service duplicates them.

The current event-schema version is 3.

* Implement service-endpoint capability for contract tests (launchdarkly#90)

* Implement service-endpoint capability for contract tests

* Ensure event summary information is reset thoroughly (#92)

* Restrict tag value length (#93)

* master -> main (#94)

* Update to cimg/rust:1.60.0 and remove nightly toolchain (#95)

* Update to cimg/rust:1.60.0

This update allows us to remove the nightly toolchain that was necessary for generating code coverage.

* ci: use large executor class

Medium+ seems to be running into memory exhaustion when linking in the test step. Reducing the number of parallel builds in the test
using -j1 did not help.

* Prevent sending shutdown broadcast when in offline mode (#97)

* Add polling capability to test harness (#99)

* [sc-169541] Fix temporary/permanent polling failure and allow for retry of initial poll. (#100)

* Bump eventsource-client to 0.11.0 (#107)

This includes support for the retry jitter and backoff reset if the
connect remains stable.

* fix script permissions

* Merge v1 into main (#117)

* Remove alias events and inline users option (launchdarkly#75)

* Add support for persistent data stores (launchdarkly#72)

This work introduces the required structure to support persistent data
stores like Redis. It includes the PersistentDataStoreWrapper and the
associated caching methods.

Along with these changes, the file structure was reshaped to more
clearly separate out store related blocks of code. Work was also done to
remove the patch and delete methods, favoring instead a single upsert
method.

Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>

* Update to Rust 2021 Edition

* Updating SDK with new context evaluation logic (launchdarkly#78)

* Update contract tests to handle contextBuild/contextConvert commands (launchdarkly#83)

* Add contextBuild command to contract-tests

* Use server-sdk's symbols instead of directly importing evaluation crate in contract-test

* Remove passing contract tests from testharness-suppressions.txt

* Update events to support contexts (launchdarkly#82)

* Update test harness suppression file with latest status (launchdarkly#86)

* Address retry behavior from contract tests (launchdarkly#87)

* Fix evaluation pathing from merge forward

* Remove literal attribute test suppression (launchdarkly#89)

* Update v1 with latest changes from master (launchdarkly#91)

* Add X-LaunchDarkly-PayloadID and X-LaunchDarkly-Event-Schema headers (launchdarkly#88)

Adds missing headers to event sending code.

The payload ID allows event recorder to dedup events if the SDK sends them multiple times (because of retries)
or if some other upstream service duplicates them.

The current event-schema version is 3.

* Implement service-endpoint capability for contract tests (launchdarkly#90)

* Rename transient back to anonymous (#96)

* Rename transient back to anonymous, using updated evaluation crate APIs.

* Fix event summary test from main merge (#98)

* Allow using hyper for polling and event sending. (#102)

Co-authored-by: Matthew M. Keeler <mkeeler@launchdarkly.com>

* Fix support for HTTPS with hyper (#104)

* Change streaming max delay to 30s (#103)

According to the [streaming spec][1], the maximum retry delay should be
set to 30 secondary.

[1]: https://launchdarkly.atlassian.net/wiki/spaces/ENG/pages/521668284/SDK+streaming+specification

* Add 1 second sleep between event sender retries (#105)

In accordance with the [event processor spec][1], we need to sleep once
after an initial failed attempt.

[1]: https://launchdarkly.atlassian.net/wiki/spaces/ENG/pages/356745688/SDK+event+processor+design

* Accept a wider range of private attribute values (#108)

Originally we were only accepting private attribute values that were
`String`s. This is limiting, especially if you want to pre-calculate the
`Reference` instances for better performance. The
`EventProcessorBuilder` now accepts any `HashSet<Into<Reference>>`.

Additionally, I modified it so that these values are converted to
`Reference` instances during the time of construction instead of during
event generation. Construction is way less frequent, so this should
provide some amount of speed improvement as well.

* Remove secondary support & update contract tests (#109)

* tests: update some more raw JSON strings to use json! macro

* contract tests: remove secondary handling

* events: remove secondary from event serialization tests; add new test for local private attribute

* Upgrade various 3rd party libs & set MSRV (#111)

* Update dependency moka from 7.1 to 9.6

* Upgrade lru from 7.2 to 8.1

* Upgrade uuid from alpha 1.0 release to 1.2.2

* Upgrade hyper-rustls from 22.1 to 23.1

* Update env_logger from 0.9 to 0.10

* contract tests: update eventsource-client from 0.10 to 0.11

* contract tests: upgrade env_logger from 0.9 to 0.10

* contract tests: upgrade actix from 0.12 to 0.13

* contract tests: bump actix-web off beta version to 4.2.1

* Update CircleCI musl executor to 1.60.0, and bump MSRV to 1.60.0

Since >= 1 of our dependencies do this now (env_logger), we should declare a
minimum supported rust version.

* hyper-rustls: enable http2 explicitely

* Update to integrate latest eval crate changes (#112)

* Integrate eval crate changes

* Fix a bunch of linter warnings (#113)

* Cleanup a batch of TODOs (#114)

* Removed todo about diagnostic event config, as it is now obsolete
* Removed multiple todos about ergonomics of detail struct (still needs to be fixed but exists as ticket)
* Removed todo about docs on ConfigBuilder, and updated those docs
* Removed todo about backoff/jitter on MockDataSourceBuilder
* Removed obsolete event tests
* Removed todo on InMemoryDataStore
* Re-Exported the 'Kind' type from the eval crate

* Bump eval crate to public 1.0.0 release (#115)

* Remove todo in streaming data source (#116)

State machine behavior to be audited at a later date.

* Remove beta warning from README (#118)

Co-authored-by: Matthew M. Keeler <mkeeler@launchdarkly.com>
Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>

Co-authored-by: Sam Stokes <sstokes@launchdarkly.com>
Co-authored-by: Harpo Roeder <hroeder@launchdarkly.com>
Co-authored-by: Ben Woskow <bwoskow@launchdarkly.com>
Co-authored-by: Ben Woskow <48036130+bwoskow-ld@users.noreply.github.com>
Co-authored-by: Ezra Stevens <estevens@launchdarkly.com>
Co-authored-by: Matthew M. Keeler <keelerm84@gmail.com>
Co-authored-by: Matthew Keeler <mkeeler@launchdarkly.com>
Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>
Co-authored-by: Casey Waldren <cwaldren@launchdarkly.com>
Co-authored-by: Eli Bishop <eli@launchdarkly.com>
LaunchDarklyReleaseBot added a commit that referenced this issue May 3, 2023
* Skeleton implementation of streaming eval - returns killswitch as flag value

* allow setting stream url in hack script

* fix default stream url

* TODO WIP handle partial lines

* flesh out buffering decoder (but wrong)

* dedupe common parsing logic between two decoder impls

* simplify with try_ready macro

* poll in a loop so we keep processing data from the stream

* Implement partial chunk buffering

* remove now-unused chunkwise decoder

* despammify debug logging a bit

* more structured error handling

* more error handling improvements

* add some tests and firm up error handling a bit

* simplify incomplete line handling, and fix eof handling

* use mem::replace to avoid copying event

* move hack script to examples/ so it can use dev-dependencies

* Clarify stream code, distinguish request errors from stream errors

* Use log crate for logging, and clean up log code

* cleanup todos

* example script: pass flags to watch as env vars

* flag eval: implement off-variation and fallthrough variation

* flag eval: implement provided default

* minor cleanup

* implement eval reasons to aid debugging

* flag eval: implement user targeting

* return our own FlagValue enum instead of exposing serde_json::Value

* define FeatureFlag struct

* strongly type more things

* parse variations into our FlagValue enum

* make creating users slightly easier

* tests for flag evaluation so far

* Implement basic rule evaluation

* support negated clauses

* add TODOs for missing implementation

* extract eventsource into a separate package

* add more TODOs

* start implementing event sending (no batching yet)

* clone() wars

* reduce verbosity by default

* make event-processor robust to tokio runtime not started yet

* add todo note

* implement a working demo with a non-tokio main thread

* implement string flags

* support null off_variation

* event sending shouldn't panic if value is None

* parse rollout (but not implemented yet)

* parse some flag rule operations (not implemented yet though)

* handle malformed 'fallthrough: {}'

* Handle patches with segment data (don't actually support segments yet though)

* Handle deletes

* Support all_flags (TODO don't send events)

* implement float and int

* clean up unimpl logging for segments

* restructure eval Detail so we can return variationIndex

* Allow user keys to be null

* missing user key should short-circuit flag eval and return default

* Log error if sending event receives non-successful response

* implement some string ops

* Implement Op::Matches

* serialize reasons and errors as IH expects

* support clauses matching against key

* implement numeric attributes and operators

* builder API for user, support name and email

* Implement all non-custom attributes

* Restore event attribute names after attribute refactoring

* Implement Deserialize for User, mostly for restwrapper

* implement config.inline_users_in_events

* don't output a bunch of tedious nulls

* Fix a couple of warnings

* implement Before and After operators

* Add circle config

* use published eventsource-client crate instead of local checkout

* Allow attributes to be null

* implement some clippy suggestions

* Implement more clippy suggestions

* and more clippy

* rejigger default handling so we can report value and default correctly in events

* got ahead of ourselves a bit

* fix clippy suggestions

* change how user inlining is serialized

* include variation index in feature event

* support i64 as attribute value

* add User.attribute mutator, and extra fun to ncurses demo

* Fake summary events (batch size = 1)

* refactor event processor to allow testing it

* use nicer json literals

* Emit reason in feature events

* Refactor client to allow (some) testing

* Allow mocking UpdateProcessor, and add overly broad test that does so

* jiggle around eval so we send events for null user keys

* Add a detail-free evaluate method that omits rather than emits reasons

* implement JSON conversions for FlagValue for restwrapper

* Remove redundant user key check

* partial impl of prereqs (TODO: events, some bugs)

* make eventsink sink events rather than JSON

* batch events, and replace hacky tokio sender with a background thread

* get rid of simplelog it is not simple

* implement flush for events

* Use batcher to generate real summary events, and transform them into the right format

* Implement rollouts

* support int attributes as well as floats, so can disallow bucketing by float

* implement segments and segmentMatch operator

* Partially implement semver operations

* fix formatting and move format check to after build

* fix clippy warnings

* fix more clippy warnings

* Remove some unnecessary tokio cruft from demo

* Make it easier to start client in a background thread

* support configuring LD urls in demo

* Declunk

* Ensure client is started exactly once

* clippy

* Rationalise client::Error

* clippy

* implement JSON and Int variations

* we clone str_variation default anyway, pass it as owned

* add non-detail variation methods for all types

* don't convert ints into floats unnecessarily

* remove boilerplate test

* dedupe users per batch

* add user agent

* forgot to add build.rs

* cargo fmt

* add stripped down readme

* add changelog contributing and license

* add .github issue and PR templates

* Bound http crate version to avoid HttpTryFrom compile errors

* dedupe user-agent construction using lazy_static

* Remove now-unused imports

* More generic user builder (#3)

* implement LD-specific semver extended semantics

* if prereq is off, return off, regardless of eval result

* ok clippy

* add identify

* implement track

* Disallow null user keys

* [ch97491] Fix clippy warnings about prefix stripping

The new version of clippy is more restrictive causing CI to fail.

* Removed the guides link

* Bump eventsource-client to 0.5.0 for decoder fixes

* okay clippy

* Return rule ID in reasons (#5)

This adds support for passing through the matched rule id

* [ch102632] use library versions of VariationIndex, Detail, Reason, Error

* [ch102632] use library versions of Flag, Segment, Rule, and friends

* [ch102632] placate clippy

* [ch102632] formatting

* Shake out implications of user.key() returning &str (#7)

Changing user.key() to return &str created a few compile errors in the SDK - this fixes them.

* Don't send feature event unless we should (#8)

Previously the Rust SDK was always sending a full "kind": "feature" event for every evaluation, in addition to the summary events. This disables those events, unless either the flag-level trackEvents property was set, or the evaluation hit a rule or fallthrough that had their respective trackEvents properties set.

I also implemented the very-special logic whereby we sometimes send reasons even if the caller didn't ask for them, and refactored the summary event logic to make it clearer that it only summarises feature events.

* don't test with int too large to represent

* example - fix unused Result warning after crate update

* Cleanup TODOs (#9)

Ditches a couple of obsolete TODOs and ensures all others (except a couple in the Circle config) are associated with CH tickets.

* Clean up exported surface area (#10)

This cleans up the SDK crate exports to just what should be needed to actually configure a client, create users and evaluate flags. It also finally brings the crate name in line with our naming conventions.

* Update tests for latest evaluation changes (#13)

* Simply URL trimming functionality (#14)

* Separate out new_unknown_event from new_feature_request (#15)

* Add contextKind designation to events (#16)

* Make custom events own struct

* Add support for alias events (#17)

* Port to newer async libraries (and compatibility with eventsource-client 0.6) (#12)

* Start implementing builders similar to the other SDKs. (#19)

* Rename update processor to data source

* Introduce data store builder pattern

* Add tests for data store (#21)

* sc-131624 Because everything inside client is Send+Sync safe now, the… (#27)

* Rlamb/sc 108602/implement client not ready (#28)

* sc-131893 Refactor starting with runtime. (#30)

* Add prereq event support (#26)

* Separate out error types (#25)

* Add version check for flag updates (#29)

* Remove test duplication and expand coverage (#33)

* SC-131531: Add code coverage & refactor config.yaml (#31)

* Update config.yml to include rust code coverage (via coverage.sh)

* Refactor config.yml

* Detail.value might be none now (#35)

With the introduction of prereq events, Detail.value might be None when
evaluating events. This is because we do not have a default value to
provide, and thus the evaluation process might yield a None result.

* Improving all_flags_detail consistency (#38)

* Update evaluate language (#37)

* Move inline_users_in_events into event processor (#39)

* Ensure client is always send + sync (#40)

* Support offline mode (#41)

* Add support for debug events (#42)

* Add version_string() function to fetch SDK version (#36)

* Fix failing tests (#44)

* Improve event processor spec alignment (#45)

The add and flush methods are not supposed to block or return success or
failure condition. However, if an event exceeds the capacity of the
event queue, we are supposed to notify the user this has occurred one
time.

Originally we were triggering flushes based off a batch size and also
some max batch deadline. The spec requires neither of these, so I have
removed them. This addresses the issue of requiring additional
configuration and takes us one step closer to alignment in this
processor.

I also did away with the separate flush channel. The spec requires only
a single inbox channel which includes messages for both events and flush
requests.

* Update eval crate name and path (#50)

* Favor error enum over String (#48)

* Add secure_mode_hash support (#46)

* Add support for user_keys_flush_interval (#47)

* Improve json f64 serialization (#49)

The default float serialization process in serde_json is fast but isn't
quite as precise as implementations in other languages. See
[this serde_json PR][1]. By enabling this feature flag, we can bring the
parsing logic inline with other language standards.

[1]: serde-rs/json#536

* Improve event processor compliance, readability, and memory (#51)

This commit addresses several issues:

- Adds support for multiple flush workers as defined by the spec
- Adds basic retry support when sending event payloads, including
  handling of retryable error codes
- Better separates out responsibilities which led to improved testing
- Provides the ability to stop the event processor thread
- Makes the event outbox a fixed size to prevent constant dynamic
  resizing
- Also makes the outbox reusable instead of re-allocating a new vector
  each time.

* Add polling support (#43)

* Remove locking when possible (#52)

We can remove locking from the event processor and the data source as it
doesn't mutate it's own state, and it is fully owned by the client.

The data store both mutates it's own internal state and is shared
between the data source and the client. We cannot remove the locking
functionality, but we can replace the Mutex with a RwLock
implementation.

We opt for using the parking_lot::RwLock instead of the
std::sync::RwLock because the built-in one on Linux suffers from writer
starvation whereas the parking_lot implementation offers a fair locking
mechanism.

* Prevent double EventProcessor::close call from blocking (#56)

* Shutdown data source on Client::close (#55)

* Update usage to reflect latest eval client side change (#59)

* Update various repo templates (#57)

* Add support for releaser (#58)

* Update documentation to prepare for upcoming release (#54)

* Update SDK to use published crates (#60)

* Final fixes for beta release (#61)

* Keyword is too long

* Add SDK contract tests (#63)

* Add creation date to alias and emit index events properly (#65)

* Fix crate repo metadata and circleci badge (#62)

* Remove dependency on OpenSSL (#64)

* Updating changelog and version for 1.0.0-beta.2 release

* Fix README in published package (#66)

crates.io only shows a README if it's at the root of the project that is
being published. To fix this, and ensure that GitHub renders the same
README, we symlink the package README to the root of the repository.

* [sc-140521] Add the ability to re-use a connector object between clients. (#68)

* Remove reliance on spectral (#69)

* Enable HTTPS support for reqwest (#70)

* Handle negative variation indexes (#71)

The SDK test harness verifies that the SDK can handle invalid flag data
(such as negative variation indexes). The evaluation crate was updated
to deal with this situation. This change brings the SDK inline with the
new evaluation types.

* Support private attributes (#67)

* Bump versions for several packages (#73)

* Add support for adding tags to all LD requests (#74)

* Update eventsource-client dependency (#76)

* Update data_source to use version 0.9.0 of eventsource-client

* Update Makefile dependencies

* Update eventsource error handling to take into account EOF

* Refactor the StreamingDataSource builder to put common building code into single function

* Update from Rust 2018 -> Rust 2021 Edition (#80)

cargo fix --edition had nothing to say.

* Bump eventsource-client to 0.10.0 (#81)

Pulls in 301 & 307 redirect handling from upstream, as well as fixed handling of Last-Event-ID.

* Add support for persistent data stores (#72) (#77)

This work introduces the required structure to support persistent data
stores like Redis. It includes the PersistentDataStoreWrapper and the
associated caching methods.

Along with these changes, the file structure was reshaped to more
clearly separate out store related blocks of code. Work was also done to
remove the patch and delete methods, favoring instead a single upsert
method.

Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>

* Pin beta versions more strictly (#84)

* Preparing 1.0.0-beta.3 (#85)

Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>

* Add X-LaunchDarkly-PayloadID and X-LaunchDarkly-Event-Schema headers (#88)

Adds missing headers to event sending code.

The payload ID allows event recorder to dedup events if the SDK sends them multiple times (because of retries)
or if some other upstream service duplicates them.

The current event-schema version is 3.

* Implement service-endpoint capability for contract tests (#90)

* Implement service-endpoint capability for contract tests

* Ensure event summary information is reset thoroughly (#92)

* Restrict tag value length (#93)

* master -> main (#94)

* Update to cimg/rust:1.60.0 and remove nightly toolchain (#95)

* Update to cimg/rust:1.60.0

This update allows us to remove the nightly toolchain that was necessary for generating code coverage.

* ci: use large executor class

Medium+ seems to be running into memory exhaustion when linking in the test step. Reducing the number of parallel builds in the test
using -j1 did not help.

* Prevent sending shutdown broadcast when in offline mode (#97)

* Add polling capability to test harness (#99)

* [sc-169541] Fix temporary/permanent polling failure and allow for retry of initial poll. (#100)

* Bump eventsource-client to 0.11.0 (#107)

This includes support for the retry jitter and backoff reset if the
connect remains stable.

* fix script permissions

* Merge v1 into main (#117)

* Remove alias events and inline users option (#75)

* Add support for persistent data stores (#72)

This work introduces the required structure to support persistent data
stores like Redis. It includes the PersistentDataStoreWrapper and the
associated caching methods.

Along with these changes, the file structure was reshaped to more
clearly separate out store related blocks of code. Work was also done to
remove the patch and delete methods, favoring instead a single upsert
method.

Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>

* Update to Rust 2021 Edition

* Updating SDK with new context evaluation logic (#78)

* Update contract tests to handle contextBuild/contextConvert commands (#83)

* Add contextBuild command to contract-tests

* Use server-sdk's symbols instead of directly importing evaluation crate in contract-test

* Remove passing contract tests from testharness-suppressions.txt

* Update events to support contexts (#82)

* Update test harness suppression file with latest status (#86)

* Address retry behavior from contract tests (#87)

* Fix evaluation pathing from merge forward

* Remove literal attribute test suppression (#89)

* Update v1 with latest changes from master (#91)

* Add X-LaunchDarkly-PayloadID and X-LaunchDarkly-Event-Schema headers (#88)

Adds missing headers to event sending code.

The payload ID allows event recorder to dedup events if the SDK sends them multiple times (because of retries)
or if some other upstream service duplicates them.

The current event-schema version is 3.

* Implement service-endpoint capability for contract tests (#90)

* Rename transient back to anonymous (#96)

* Rename transient back to anonymous, using updated evaluation crate APIs.

* Fix event summary test from main merge (#98)

* Allow using hyper for polling and event sending. (#102)

Co-authored-by: Matthew M. Keeler <mkeeler@launchdarkly.com>

* Fix support for HTTPS with hyper (#104)

* Change streaming max delay to 30s (#103)

According to the [streaming spec][1], the maximum retry delay should be
set to 30 secondary.

[1]: https://launchdarkly.atlassian.net/wiki/spaces/ENG/pages/521668284/SDK+streaming+specification

* Add 1 second sleep between event sender retries (#105)

In accordance with the [event processor spec][1], we need to sleep once
after an initial failed attempt.

[1]: https://launchdarkly.atlassian.net/wiki/spaces/ENG/pages/356745688/SDK+event+processor+design

* Accept a wider range of private attribute values (#108)

Originally we were only accepting private attribute values that were
`String`s. This is limiting, especially if you want to pre-calculate the
`Reference` instances for better performance. The
`EventProcessorBuilder` now accepts any `HashSet<Into<Reference>>`.

Additionally, I modified it so that these values are converted to
`Reference` instances during the time of construction instead of during
event generation. Construction is way less frequent, so this should
provide some amount of speed improvement as well.

* Remove secondary support & update contract tests (#109)

* tests: update some more raw JSON strings to use json! macro

* contract tests: remove secondary handling

* events: remove secondary from event serialization tests; add new test for local private attribute

* Upgrade various 3rd party libs & set MSRV (#111)

* Update dependency moka from 7.1 to 9.6

* Upgrade lru from 7.2 to 8.1

* Upgrade uuid from alpha 1.0 release to 1.2.2

* Upgrade hyper-rustls from 22.1 to 23.1

* Update env_logger from 0.9 to 0.10

* contract tests: update eventsource-client from 0.10 to 0.11

* contract tests: upgrade env_logger from 0.9 to 0.10

* contract tests: upgrade actix from 0.12 to 0.13

* contract tests: bump actix-web off beta version to 4.2.1

* Update CircleCI musl executor to 1.60.0, and bump MSRV to 1.60.0

Since >= 1 of our dependencies do this now (env_logger), we should declare a
minimum supported rust version.

* hyper-rustls: enable http2 explicitely

* Update to integrate latest eval crate changes (#112)

* Integrate eval crate changes

* Fix a bunch of linter warnings (#113)

* Cleanup a batch of TODOs (#114)

* Removed todo about diagnostic event config, as it is now obsolete
* Removed multiple todos about ergonomics of detail struct (still needs to be fixed but exists as ticket)
* Removed todo about docs on ConfigBuilder, and updated those docs
* Removed todo about backoff/jitter on MockDataSourceBuilder
* Removed obsolete event tests
* Removed todo on InMemoryDataStore
* Re-Exported the 'Kind' type from the eval crate

* Bump eval crate to public 1.0.0 release (#115)

* Remove todo in streaming data source (#116)

State machine behavior to be audited at a later date.

* Remove beta warning from README (#118)

Co-authored-by: Matthew M. Keeler <mkeeler@launchdarkly.com>
Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>

* Log persistent store error in PersistentDataStoreWrapper's init method, if cache is not infinite (#119)

* Surface persistent store error in init() for in all cases

This change ensures the any persistent store error is logged if the DataStore init() method
encounters an error.

Before, the error would only be logged if the PersistentDataStoreWrapper's cache was non-expiring.

* chore: bump cimg/rust to 1.63.0 (#121)

* chore: bump cimg/rust to 1.63.0

* chore: bump cimg/rust to 1.63.0

* Bump MSRV to 1.63.0 (#122)

* fix: Read entire polling response body (#124)

Our usage of the hyper client is incorrect. In the current format, it
will only read the first chunk of a response (see [`data` method][1]).
Apparently this has proven sufficient to pass our internal testing thus
far. However, recent failures through IH have uncovered this deficiency.

Now we are exceeding that default threshold and spilling over into a
second streaming chunk, thereby uncovering the flaw in our usage.

To address this, I am updating the usage to read the entire body before
returning so that we always have the full payload before we attempt any
JSON parsing.

[1]: https://docs.rs/hyper/latest/hyper/body/struct.Body.html

* chore: Bump MSRV to 1.64.0 (#122) (#36) (#125)

* Merge from public (#126)

* chore: Bump MSRV to 1.64.0 (#122) (#36)

* fix: Fix releaser docker image override (#37)

---------

Co-authored-by: Sam Stokes <sstokes@launchdarkly.com>
Co-authored-by: Harpo Roeder <hroeder@launchdarkly.com>
Co-authored-by: Ben Woskow <bwoskow@launchdarkly.com>
Co-authored-by: Ben Woskow <48036130+bwoskow-ld@users.noreply.github.com>
Co-authored-by: Ezra Stevens <estevens@launchdarkly.com>
Co-authored-by: Matthew M. Keeler <keelerm84@gmail.com>
Co-authored-by: Matthew Keeler <mkeeler@launchdarkly.com>
Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>
Co-authored-by: Casey Waldren <cwaldren@launchdarkly.com>
Co-authored-by: Eli Bishop <eli@launchdarkly.com>
Co-authored-by: LaunchDarklyReleaseBot <launchdarklyreleasebot@launchdarkly.com>
Co-authored-by: Louis Chan <lchan@launchdarkly.com>
Co-authored-by: Louis Chan <91093020+louis-launchdarkly@users.noreply.github.com>
LaunchDarklyReleaseBot added a commit that referenced this issue May 10, 2023
* flesh out buffering decoder (but wrong)

* dedupe common parsing logic between two decoder impls

* simplify with try_ready macro

* poll in a loop so we keep processing data from the stream

* Implement partial chunk buffering

* remove now-unused chunkwise decoder

* despammify debug logging a bit

* more structured error handling

* more error handling improvements

* add some tests and firm up error handling a bit

* simplify incomplete line handling, and fix eof handling

* use mem::replace to avoid copying event

* move hack script to examples/ so it can use dev-dependencies

* Clarify stream code, distinguish request errors from stream errors

* Use log crate for logging, and clean up log code

* cleanup todos

* example script: pass flags to watch as env vars

* flag eval: implement off-variation and fallthrough variation

* flag eval: implement provided default

* minor cleanup

* implement eval reasons to aid debugging

* flag eval: implement user targeting

* return our own FlagValue enum instead of exposing serde_json::Value

* define FeatureFlag struct

* strongly type more things

* parse variations into our FlagValue enum

* make creating users slightly easier

* tests for flag evaluation so far

* Implement basic rule evaluation

* support negated clauses

* add TODOs for missing implementation

* extract eventsource into a separate package

* add more TODOs

* start implementing event sending (no batching yet)

* clone() wars

* reduce verbosity by default

* make event-processor robust to tokio runtime not started yet

* add todo note

* implement a working demo with a non-tokio main thread

* implement string flags

* support null off_variation

* event sending shouldn't panic if value is None

* parse rollout (but not implemented yet)

* parse some flag rule operations (not implemented yet though)

* handle malformed 'fallthrough: {}'

* Handle patches with segment data (don't actually support segments yet though)

* Handle deletes

* Support all_flags (TODO don't send events)

* implement float and int

* clean up unimpl logging for segments

* restructure eval Detail so we can return variationIndex

* Allow user keys to be null

* missing user key should short-circuit flag eval and return default

* Log error if sending event receives non-successful response

* implement some string ops

* Implement Op::Matches

* serialize reasons and errors as IH expects

* support clauses matching against key

* implement numeric attributes and operators

* builder API for user, support name and email

* Implement all non-custom attributes

* Restore event attribute names after attribute refactoring

* Implement Deserialize for User, mostly for restwrapper

* implement config.inline_users_in_events

* don't output a bunch of tedious nulls

* Fix a couple of warnings

* implement Before and After operators

* Add circle config

* use published eventsource-client crate instead of local checkout

* Allow attributes to be null

* implement some clippy suggestions

* Implement more clippy suggestions

* and more clippy

* rejigger default handling so we can report value and default correctly in events

* got ahead of ourselves a bit

* fix clippy suggestions

* change how user inlining is serialized

* include variation index in feature event

* support i64 as attribute value

* add User.attribute mutator, and extra fun to ncurses demo

* Fake summary events (batch size = 1)

* refactor event processor to allow testing it

* use nicer json literals

* Emit reason in feature events

* Refactor client to allow (some) testing

* Allow mocking UpdateProcessor, and add overly broad test that does so

* jiggle around eval so we send events for null user keys

* Add a detail-free evaluate method that omits rather than emits reasons

* implement JSON conversions for FlagValue for restwrapper

* Remove redundant user key check

* partial impl of prereqs (TODO: events, some bugs)

* make eventsink sink events rather than JSON

* batch events, and replace hacky tokio sender with a background thread

* get rid of simplelog it is not simple

* implement flush for events

* Use batcher to generate real summary events, and transform them into the right format

* Implement rollouts

* support int attributes as well as floats, so can disallow bucketing by float

* implement segments and segmentMatch operator

* Partially implement semver operations

* fix formatting and move format check to after build

* fix clippy warnings

* fix more clippy warnings

* Remove some unnecessary tokio cruft from demo

* Make it easier to start client in a background thread

* support configuring LD urls in demo

* Declunk

* Ensure client is started exactly once

* clippy

* Rationalise client::Error

* clippy

* implement JSON and Int variations

* we clone str_variation default anyway, pass it as owned

* add non-detail variation methods for all types

* don't convert ints into floats unnecessarily

* remove boilerplate test

* dedupe users per batch

* add user agent

* forgot to add build.rs

* cargo fmt

* add stripped down readme

* add changelog contributing and license

* add .github issue and PR templates

* Bound http crate version to avoid HttpTryFrom compile errors

* dedupe user-agent construction using lazy_static

* Remove now-unused imports

* More generic user builder (#3)

* implement LD-specific semver extended semantics

* if prereq is off, return off, regardless of eval result

* ok clippy

* add identify

* implement track

* Disallow null user keys

* [ch97491] Fix clippy warnings about prefix stripping

The new version of clippy is more restrictive causing CI to fail.

* Removed the guides link

* Bump eventsource-client to 0.5.0 for decoder fixes

* okay clippy

* Return rule ID in reasons (#5)

This adds support for passing through the matched rule id

* [ch102632] use library versions of VariationIndex, Detail, Reason, Error

* [ch102632] use library versions of Flag, Segment, Rule, and friends

* [ch102632] placate clippy

* [ch102632] formatting

* Shake out implications of user.key() returning &str (#7)

Changing user.key() to return &str created a few compile errors in the SDK - this fixes them.

* Don't send feature event unless we should (#8)

Previously the Rust SDK was always sending a full "kind": "feature" event for every evaluation, in addition to the summary events. This disables those events, unless either the flag-level trackEvents property was set, or the evaluation hit a rule or fallthrough that had their respective trackEvents properties set.

I also implemented the very-special logic whereby we sometimes send reasons even if the caller didn't ask for them, and refactored the summary event logic to make it clearer that it only summarises feature events.

* don't test with int too large to represent

* example - fix unused Result warning after crate update

* Cleanup TODOs (#9)

Ditches a couple of obsolete TODOs and ensures all others (except a couple in the Circle config) are associated with CH tickets.

* Clean up exported surface area (#10)

This cleans up the SDK crate exports to just what should be needed to actually configure a client, create users and evaluate flags. It also finally brings the crate name in line with our naming conventions.

* Update tests for latest evaluation changes (#13)

* Simply URL trimming functionality (#14)

* Separate out new_unknown_event from new_feature_request (#15)

* Add contextKind designation to events (#16)

* Make custom events own struct

* Add support for alias events (#17)

* Port to newer async libraries (and compatibility with eventsource-client 0.6) (#12)

* Start implementing builders similar to the other SDKs. (#19)

* Rename update processor to data source

* Introduce data store builder pattern

* Add tests for data store (#21)

* sc-131624 Because everything inside client is Send+Sync safe now, the… (#27)

* Rlamb/sc 108602/implement client not ready (#28)

* sc-131893 Refactor starting with runtime. (#30)

* Add prereq event support (#26)

* Separate out error types (#25)

* Add version check for flag updates (#29)

* Remove test duplication and expand coverage (#33)

* SC-131531: Add code coverage & refactor config.yaml (#31)

* Update config.yml to include rust code coverage (via coverage.sh)

* Refactor config.yml

* Detail.value might be none now (#35)

With the introduction of prereq events, Detail.value might be None when
evaluating events. This is because we do not have a default value to
provide, and thus the evaluation process might yield a None result.

* Improving all_flags_detail consistency (#38)

* Update evaluate language (#37)

* Move inline_users_in_events into event processor (#39)

* Ensure client is always send + sync (#40)

* Support offline mode (#41)

* Add support for debug events (#42)

* Add version_string() function to fetch SDK version (#36)

* Fix failing tests (#44)

* Improve event processor spec alignment (#45)

The add and flush methods are not supposed to block or return success or
failure condition. However, if an event exceeds the capacity of the
event queue, we are supposed to notify the user this has occurred one
time.

Originally we were triggering flushes based off a batch size and also
some max batch deadline. The spec requires neither of these, so I have
removed them. This addresses the issue of requiring additional
configuration and takes us one step closer to alignment in this
processor.

I also did away with the separate flush channel. The spec requires only
a single inbox channel which includes messages for both events and flush
requests.

* Update eval crate name and path (#50)

* Favor error enum over String (#48)

* Add secure_mode_hash support (#46)

* Add support for user_keys_flush_interval (#47)

* Improve json f64 serialization (#49)

The default float serialization process in serde_json is fast but isn't
quite as precise as implementations in other languages. See
[this serde_json PR][1]. By enabling this feature flag, we can bring the
parsing logic inline with other language standards.

[1]: serde-rs/json#536

* Improve event processor compliance, readability, and memory (#51)

This commit addresses several issues:

- Adds support for multiple flush workers as defined by the spec
- Adds basic retry support when sending event payloads, including
  handling of retryable error codes
- Better separates out responsibilities which led to improved testing
- Provides the ability to stop the event processor thread
- Makes the event outbox a fixed size to prevent constant dynamic
  resizing
- Also makes the outbox reusable instead of re-allocating a new vector
  each time.

* Add polling support (#43)

* Remove locking when possible (#52)

We can remove locking from the event processor and the data source as it
doesn't mutate it's own state, and it is fully owned by the client.

The data store both mutates it's own internal state and is shared
between the data source and the client. We cannot remove the locking
functionality, but we can replace the Mutex with a RwLock
implementation.

We opt for using the parking_lot::RwLock instead of the
std::sync::RwLock because the built-in one on Linux suffers from writer
starvation whereas the parking_lot implementation offers a fair locking
mechanism.

* Prevent double EventProcessor::close call from blocking (#56)

* Shutdown data source on Client::close (#55)

* Update usage to reflect latest eval client side change (#59)

* Update various repo templates (#57)

* Add support for releaser (#58)

* Update documentation to prepare for upcoming release (#54)

* Update SDK to use published crates (#60)

* Final fixes for beta release (#61)

* Keyword is too long

* Add SDK contract tests (#63)

* Add creation date to alias and emit index events properly (#65)

* Fix crate repo metadata and circleci badge (#62)

* Remove dependency on OpenSSL (#64)

* Updating changelog and version for 1.0.0-beta.2 release

* Fix README in published package (#66)

crates.io only shows a README if it's at the root of the project that is
being published. To fix this, and ensure that GitHub renders the same
README, we symlink the package README to the root of the repository.

* [sc-140521] Add the ability to re-use a connector object between clients. (#68)

* Remove reliance on spectral (#69)

* Enable HTTPS support for reqwest (#70)

* Handle negative variation indexes (#71)

The SDK test harness verifies that the SDK can handle invalid flag data
(such as negative variation indexes). The evaluation crate was updated
to deal with this situation. This change brings the SDK inline with the
new evaluation types.

* Support private attributes (#67)

* Bump versions for several packages (#73)

* Add support for adding tags to all LD requests (#74)

* Update eventsource-client dependency (#76)

* Update data_source to use version 0.9.0 of eventsource-client

* Update Makefile dependencies

* Update eventsource error handling to take into account EOF

* Refactor the StreamingDataSource builder to put common building code into single function

* Update from Rust 2018 -> Rust 2021 Edition (#80)

cargo fix --edition had nothing to say.

* Bump eventsource-client to 0.10.0 (#81)

Pulls in 301 & 307 redirect handling from upstream, as well as fixed handling of Last-Event-ID.

* Add support for persistent data stores (#72) (#77)

This work introduces the required structure to support persistent data
stores like Redis. It includes the PersistentDataStoreWrapper and the
associated caching methods.

Along with these changes, the file structure was reshaped to more
clearly separate out store related blocks of code. Work was also done to
remove the patch and delete methods, favoring instead a single upsert
method.

Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>

* Pin beta versions more strictly (#84)

* Preparing 1.0.0-beta.3 (#85)

Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>

* Add X-LaunchDarkly-PayloadID and X-LaunchDarkly-Event-Schema headers (#88)

Adds missing headers to event sending code.

The payload ID allows event recorder to dedup events if the SDK sends them multiple times (because of retries)
or if some other upstream service duplicates them.

The current event-schema version is 3.

* Implement service-endpoint capability for contract tests (#90)

* Implement service-endpoint capability for contract tests

* Ensure event summary information is reset thoroughly (#92)

* Restrict tag value length (#93)

* master -> main (#94)

* Update to cimg/rust:1.60.0 and remove nightly toolchain (#95)

* Update to cimg/rust:1.60.0

This update allows us to remove the nightly toolchain that was necessary for generating code coverage.

* ci: use large executor class

Medium+ seems to be running into memory exhaustion when linking in the test step. Reducing the number of parallel builds in the test
using -j1 did not help.

* Prevent sending shutdown broadcast when in offline mode (#97)

* Add polling capability to test harness (#99)

* [sc-169541] Fix temporary/permanent polling failure and allow for retry of initial poll. (#100)

* Bump eventsource-client to 0.11.0 (#107)

This includes support for the retry jitter and backoff reset if the
connect remains stable.

* fix script permissions

* Merge v1 into main (#117)

* Remove alias events and inline users option (#75)

* Add support for persistent data stores (#72)

This work introduces the required structure to support persistent data
stores like Redis. It includes the PersistentDataStoreWrapper and the
associated caching methods.

Along with these changes, the file structure was reshaped to more
clearly separate out store related blocks of code. Work was also done to
remove the patch and delete methods, favoring instead a single upsert
method.

Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>

* Update to Rust 2021 Edition

* Updating SDK with new context evaluation logic (#78)

* Update contract tests to handle contextBuild/contextConvert commands (#83)

* Add contextBuild command to contract-tests

* Use server-sdk's symbols instead of directly importing evaluation crate in contract-test

* Remove passing contract tests from testharness-suppressions.txt

* Update events to support contexts (#82)

* Update test harness suppression file with latest status (#86)

* Address retry behavior from contract tests (#87)

* Fix evaluation pathing from merge forward

* Remove literal attribute test suppression (#89)

* Update v1 with latest changes from master (#91)

* Add X-LaunchDarkly-PayloadID and X-LaunchDarkly-Event-Schema headers (#88)

Adds missing headers to event sending code.

The payload ID allows event recorder to dedup events if the SDK sends them multiple times (because of retries)
or if some other upstream service duplicates them.

The current event-schema version is 3.

* Implement service-endpoint capability for contract tests (#90)

* Rename transient back to anonymous (#96)

* Rename transient back to anonymous, using updated evaluation crate APIs.

* Fix event summary test from main merge (#98)

* Allow using hyper for polling and event sending. (#102)

Co-authored-by: Matthew M. Keeler <mkeeler@launchdarkly.com>

* Fix support for HTTPS with hyper (#104)

* Change streaming max delay to 30s (#103)

According to the [streaming spec][1], the maximum retry delay should be
set to 30 secondary.

[1]: https://launchdarkly.atlassian.net/wiki/spaces/ENG/pages/521668284/SDK+streaming+specification

* Add 1 second sleep between event sender retries (#105)

In accordance with the [event processor spec][1], we need to sleep once
after an initial failed attempt.

[1]: https://launchdarkly.atlassian.net/wiki/spaces/ENG/pages/356745688/SDK+event+processor+design

* Accept a wider range of private attribute values (#108)

Originally we were only accepting private attribute values that were
`String`s. This is limiting, especially if you want to pre-calculate the
`Reference` instances for better performance. The
`EventProcessorBuilder` now accepts any `HashSet<Into<Reference>>`.

Additionally, I modified it so that these values are converted to
`Reference` instances during the time of construction instead of during
event generation. Construction is way less frequent, so this should
provide some amount of speed improvement as well.

* Remove secondary support & update contract tests (#109)

* tests: update some more raw JSON strings to use json! macro

* contract tests: remove secondary handling

* events: remove secondary from event serialization tests; add new test for local private attribute

* Upgrade various 3rd party libs & set MSRV (#111)

* Update dependency moka from 7.1 to 9.6

* Upgrade lru from 7.2 to 8.1

* Upgrade uuid from alpha 1.0 release to 1.2.2

* Upgrade hyper-rustls from 22.1 to 23.1

* Update env_logger from 0.9 to 0.10

* contract tests: update eventsource-client from 0.10 to 0.11

* contract tests: upgrade env_logger from 0.9 to 0.10

* contract tests: upgrade actix from 0.12 to 0.13

* contract tests: bump actix-web off beta version to 4.2.1

* Update CircleCI musl executor to 1.60.0, and bump MSRV to 1.60.0

Since >= 1 of our dependencies do this now (env_logger), we should declare a
minimum supported rust version.

* hyper-rustls: enable http2 explicitely

* Update to integrate latest eval crate changes (#112)

* Integrate eval crate changes

* Fix a bunch of linter warnings (#113)

* Cleanup a batch of TODOs (#114)

* Removed todo about diagnostic event config, as it is now obsolete
* Removed multiple todos about ergonomics of detail struct (still needs to be fixed but exists as ticket)
* Removed todo about docs on ConfigBuilder, and updated those docs
* Removed todo about backoff/jitter on MockDataSourceBuilder
* Removed obsolete event tests
* Removed todo on InMemoryDataStore
* Re-Exported the 'Kind' type from the eval crate

* Bump eval crate to public 1.0.0 release (#115)

* Remove todo in streaming data source (#116)

State machine behavior to be audited at a later date.

* Remove beta warning from README (#118)

Co-authored-by: Matthew M. Keeler <mkeeler@launchdarkly.com>
Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>

* Log persistent store error in PersistentDataStoreWrapper's init method, if cache is not infinite (#119)

* Surface persistent store error in init() for in all cases

This change ensures the any persistent store error is logged if the DataStore init() method
encounters an error.

Before, the error would only be logged if the PersistentDataStoreWrapper's cache was non-expiring.

* chore: bump cimg/rust to 1.63.0 (#121)

* chore: bump cimg/rust to 1.63.0

* chore: bump cimg/rust to 1.63.0

* Bump MSRV to 1.63.0 (#122)

* fix: Read entire polling response body (#124)

Our usage of the hyper client is incorrect. In the current format, it
will only read the first chunk of a response (see [`data` method][1]).
Apparently this has proven sufficient to pass our internal testing thus
far. However, recent failures through IH have uncovered this deficiency.

Now we are exceeding that default threshold and spilling over into a
second streaming chunk, thereby uncovering the flaw in our usage.

To address this, I am updating the usage to read the entire body before
returning so that we always have the full payload before we attempt any
JSON parsing.

[1]: https://docs.rs/hyper/latest/hyper/body/struct.Body.html

* chore: Bump MSRV to 1.64.0 (#122) (#36) (#125)

* Merge from public (#126)

* chore: Bump MSRV to 1.64.0 (#122) (#36)

* fix: Fix releaser docker image override (#37)

* fix: Fix secure mode hashing algorithm (#127)

The secure mode hash should make use of the context's fully qualified
key. This commit updates that usage and also bumps the eval crate as the
canonical key encoding solution was incorrect as well.

Additionally, the contract test service has been expanded to support the
secure mode hash capability so we can ensure compliance going forward.

* ci: Bump MSRV version to eliminate build fails (#128)

---------

Co-authored-by: Sam Stokes <sstokes@launchdarkly.com>
Co-authored-by: Harpo Roeder <hroeder@launchdarkly.com>
Co-authored-by: Ben Woskow <bwoskow@launchdarkly.com>
Co-authored-by: Ben Woskow <48036130+bwoskow-ld@users.noreply.github.com>
Co-authored-by: Ezra Stevens <estevens@launchdarkly.com>
Co-authored-by: Matthew M. Keeler <keelerm84@gmail.com>
Co-authored-by: Matthew Keeler <mkeeler@launchdarkly.com>
Co-authored-by: Ryan Lamb <4955475+kinyoklion@users.noreply.github.com>
Co-authored-by: Casey Waldren <cwaldren@launchdarkly.com>
Co-authored-by: Eli Bishop <eli@launchdarkly.com>
Co-authored-by: LaunchDarklyReleaseBot <launchdarklyreleasebot@launchdarkly.com>
Co-authored-by: Louis Chan <lchan@launchdarkly.com>
Co-authored-by: Louis Chan <91093020+louis-launchdarkly@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants