test(ofrep): adopt the OpenFeature Provider TCK - #1840
Draft
aepfli wants to merge 17 commits into
Draft
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueComment |
aepfli
force-pushed
the
feat/provider-tck-ofrep
branch
from
September 11, 2026 10:36
f51b3f1 to
a53854b
Compare
aepfli
changed the base branch from
feat/provider-tck
to
feat/provider-tck-flagd
September 11, 2026 10:36
github-actions
Bot
requested review from
Kavindu-Dodan,
beeme1mr,
thisthat and
toddbaert
September 11, 2026 10:36
aepfli
force-pushed
the
feat/provider-tck-flagd
branch
from
September 11, 2026 10:49
3180328 to
20cf419
Compare
aepfli
force-pushed
the
feat/provider-tck-ofrep
branch
from
September 11, 2026 10:50
a53854b to
36be24c
Compare
aepfli
force-pushed
the
feat/provider-tck-flagd
branch
from
September 11, 2026 13:44
20cf419 to
177ebe0
Compare
aepfli
force-pushed
the
feat/provider-tck-ofrep
branch
from
September 11, 2026 13:44
36be24c to
5be8740
Compare
aepfli
force-pushed
the
feat/provider-tck-flagd
branch
from
September 11, 2026 16:18
177ebe0 to
5849212
Compare
aepfli
force-pushed
the
feat/provider-tck-ofrep
branch
from
September 11, 2026 16:18
5be8740 to
83d0cc4
Compare
aepfli
force-pushed
the
feat/provider-tck-flagd
branch
from
September 11, 2026 17:13
5849212 to
76f441a
Compare
aepfli
force-pushed
the
feat/provider-tck-ofrep
branch
from
September 11, 2026 17:13
83d0cc4 to
acadba1
Compare
aepfli
force-pushed
the
feat/provider-tck-flagd
branch
from
September 11, 2026 17:17
76f441a to
e120ae2
Compare
aepfli
force-pushed
the
feat/provider-tck-ofrep
branch
2 times, most recently
from
September 12, 2026 06:42
660cc32 to
1036a65
Compare
aepfli
force-pushed
the
feat/provider-tck-flagd
branch
2 times, most recently
from
September 12, 2026 10:47
5325453 to
74f2c67
Compare
aepfli
force-pushed
the
feat/provider-tck-ofrep
branch
from
September 12, 2026 10:47
1036a65 to
d7a8c3a
Compare
aepfli
force-pushed
the
feat/provider-tck-flagd
branch
from
September 12, 2026 13:47
74f2c67 to
4ba9735
Compare
aepfli
force-pushed
the
feat/provider-tck-ofrep
branch
from
September 12, 2026 13:47
d7a8c3a to
27ff382
Compare
aepfli
force-pushed
the
feat/provider-tck-flagd
branch
from
September 13, 2026 08:48
fdbe6a8 to
8ac8f62
Compare
aepfli
force-pushed
the
feat/provider-tck-ofrep
branch
from
September 13, 2026 08:48
c826be5 to
08fa51a
Compare
aepfli
force-pushed
the
feat/provider-tck-flagd
branch
from
September 13, 2026 09:09
8ac8f62 to
a0bd1eb
Compare
aepfli
force-pushed
the
feat/provider-tck-ofrep
branch
from
September 13, 2026 09:09
08fa51a to
94d2c35
Compare
aepfli
force-pushed
the
feat/provider-tck-flagd
branch
from
September 13, 2026 14:48
a0bd1eb to
c7c4569
Compare
aepfli
force-pushed
the
feat/provider-tck-ofrep
branch
from
September 13, 2026 14:48
94d2c35 to
a447b4c
Compare
aepfli
force-pushed
the
feat/provider-tck-flagd
branch
from
September 13, 2026 18:03
c7c4569 to
d7a1c24
Compare
aepfli
force-pushed
the
feat/provider-tck-ofrep
branch
from
September 13, 2026 18:03
a447b4c to
435edfb
Compare
aepfli
force-pushed
the
feat/provider-tck-flagd
branch
from
September 13, 2026 19:22
d7a1c24 to
1e3c1dc
Compare
aepfli
force-pushed
the
feat/provider-tck-ofrep
branch
from
September 13, 2026 19:23
435edfb to
b001ced
Compare
OFREP is a protocol, not a vendor, so the suite needs no new infrastructure: flagd already serves the OFREP HTTP API on 8016 inside the flagd-testbed image that the flagd TCK suites use, alongside the launchpad control API on 8080. The Compose stack is therefore the same image with a different port exposed, and the whole adoption is one test class plus one dependency. Four capabilities are withheld, all traceable to the same fact: OfrepProvider implements FeatureProvider rather than extending EventProvider and overrides no lifecycle method, so it has no state, no stream, no poll loop and no initialize(). It cannot emit events (EVENTS), cannot observe the backend going away (STALE) or changing (CONFIGURATION_CHANGE), and cannot fail initialisation against a dead port (UNAVAILABLE_INIT). Each omission is justified against specific lines of the provider in the capabilities() javadoc. events.feature and lifecycle.feature are both tagged @events at feature level, so 5 scenarios are reported as skipped and 24 run. OBJECT and STRICT_NUMERIC_TYPING are both declared. Unlike the flagd provider, OFREP does not silently narrow a float to an integer: values are deserialised by a plain Jackson ObjectMapper into an untyped Object, so a JSON fraction arrives as Double and a JSON integer as Integer, and handleResolved admits a value only on an exact type.isInstance check. float-flag requested as an integer is reported as TYPE_MISMATCH with the code default rather than truncated to 0. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
The base renamed the containerised base class, retired @strict-numeric-typing in favour of @numeric-coercion, and now refuses a declaration that names a reserved or not-applicable capability. EnumSet.complementOf swept up @large-integers, @targeting and @caching, so the suite would have stopped at startup; Capability.declarableExcept leaves those out on its own. LIFECYCLE is withheld as well, which the complement had quietly claimed since the capability appeared: OfrepProvider has no initialize(), so the readiness scenario passed exactly as it does for NoOpProvider, and the new shutdown scenarios gated by the same tag are skipped rather than passed vacuously. NUMERIC_COERCION is withheld because the tag now requires the lossless direction too, and Resolver.handleResolved admits a value only on an exact type.isInstance check: integer-flag requested as a float arrives from Jackson as an Integer and is refused. Strict typing in both directions is a choice under the suite's model, not a defect, so no KnownDeviation goes with it. Read from the source, not from a run. The class Javadoc records that flagd-testbed v3.8.0 serves none of the six new canonical flags, so the four untagged scenarios that read them fail FLAG_NOT_FOUND until the testbed is updated; that is the stack's gap, not the provider's, so it is documented rather than declared. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
…tbed lacks The same note the flagd adoption carries, next to the image tag it is about: flagd-testbed v3.8.0 serves none of the six flags the bumped assets added, so the untagged scenarios that read them fail FLAG_NOT_FOUND until the testbed is updated, and the tag here is what to bump when it is. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
…vidence The falsy-flag rename in the base removes three failures here without touching the provider: flagd-testbed already served boolean-zero-flag, integer-zero-flag and string-zero-flag with zero/non-zero variants, and spec ba002ce8 moved the canonical names onto the testbed's rather than the other way round. The suite goes from four untagged failures to one. The notes now say what is actually missing -- large-integer-flag, and only that, because huge-integer-flag sits behind @large-integers and integral-float-flag behind @numeric-coercion, neither of which this provider declares. @numeric-coercion stays withheld, now on stated evidence rather than a shorter argument. Go's OFREP provider declares it and this one does not, which looked like an unexamined declaration on one side; it is not. handleResolved admits a value only on an exact type.isInstance check (Resolver.java:183-191) with no integral check and no round trip anywhere in the path, so of the tag's three scenarios this provider passes one: the lossy case is right for the wrong reason, and both lossless cases fail, integer-flag requested as a float and integral-float-flag requested as an integer alike. Declaring it would turn two scenarios red -- three, counting that the testbed cannot serve integral-float-flag at all. Go coerces and this does not; the two declarations describe two implementations, not one protocol, which is possible precisely because OFREP is JSON and integer-ness is the provider's decision. No KnownDeviation accompanies it, and that is a decision rather than silence. Appendix F is explicit that this is the one capability the specification does not define, that its rule is borrowed from flagd's ADR, and that "a provider that behaves differently is not violating the specification" -- having retracted an earlier draft that called non-declaration an admission of a known bug. A deviation entry would assert a defect the spec says is not one: the opposite mistake from a vacuous declaration, in the same currency. What the entry does record is that the reasoning is source-derived, that no unit test pins the numeric pair, and what a run would have to show for the declaration to change. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
The base gates "A provider that was shut down can be initialized again" on @reinitialization, because Requirement 2.5.2 permits reuse after shutdown rather than requiring it. declarableExcept(...) hands the new tag out by default, so a provider that cannot be restarted has to say so or it publishes a claim nothing examined -- the one failure mode this declaration exists to prevent. OfrepProvider cannot be restarted. shutdown() terminates the executor the HTTP client runs on (OfrepProvider.java:90-108) and the class overrides no initialize(), so nothing recreates it. That is the choice 2.5.2 offers rather than a defect, and no KnownDeviation accompanies the omission. It changes no result on its own: the scenario carries @lifecycle too, which this provider already withholds because it has no initialisation to observe, so the skip was happening either way. The declaration is what stops being a half-truth. The @lifecycle bullet no longer counts re-initialisation among the scenarios that tag gates, since it does not any more. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
The TCK no longer sets @large-integers apart as "not applicable in Java", so declarableExcept(...) no longer leaves it out on its own and this suite names it alongside the six capabilities it already withholds. The run is unchanged: the scenario was skipped before and is skipped now, because the SDK's integer accessor is a 32-bit Integer and 2^53 - 1 has no room in it. That is a fact about the SDK rather than about OFREP, which is why it is recorded once in Appendix F and needs no knownDeviations entry here -- unlike the numeric-coercion gap, which is the provider's own. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
@Variants and @targeting arrived with the base's submodule bump and declarableExcept picks both up, so the declaration grew by two capabilities while the javadoc that argues every withheld tag at length said nothing about either. One run against flagd-testbed v3.8.0 through the OFREP HTTP API: 38 pass, 12 skipped, 2 failed. @targeting is worth more here than its name suggests. The provider sends the evaluation context in the request body and the backend evaluates the rule, so targeting-key-flag's three scenarios are the only ones in the canonical set that would notice a context dropped on the way out -- every other flag resolves the same way with or without one. All three pass. @Variants passes seven of its eight rows. The eighth asks for large-integer-flag's max-int32 and is answered with no variant, because v3.8.0 does not serve that flag at all -- the gap the Compose header already records for the untagged precision scenario, now reached twice rather than once, so "one untagged scenario" is a scenario short. Withholding the tag would hide both failures behind a claim about the provider that the run does not support. The complementOf paragraph called @targeting reserved alongside @caching. It is declarable now; the argument survives with one tag, because what makes the form of the call right is not how large the reserved set happens to be. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
Measured, not assumed, and the measurement overturned the expectation it was made to confirm. Declared, all four rows of the new outline fail: 56 scenarios, 38 passing, 12 skipped, 6 failed, the four extra failures beyond the two testbed ones all reading "expected: null but was: FLAG_NOT_FOUND". The value assertion passes, because the provider returns the code default on an error -- right answer, wrong reason. The expectation was that this is architectural: the caller's default never leaves the process, so a backend cannot return it. That is not what the backend does. flagd's OFREP endpoint answers a disabled flag with 200, reason DISABLED and no value member, probed directly against the pinned v3.8.0 image. OFREP's evaluationSuccess requires only key and reason; value is not required, because one shape a success may take is codeDefaultFlag -- "This schema has no value property. The provider must use the code default value when processing this response." DISABLED is in the reason enum. The response is well-formed and says exactly what the scenario asserts. So this is a provider gap against a MUST in the protocol the provider implements, and it is wider than the rows that found it: every codeDefaultFlag response reaches the application as FLAG_NOT_FOUND, so an application checking the error code sees a failure on an evaluation that succeeded. handleResolved treats a null value as an absent flag and discards the reason it parsed a field earlier. The tag stays withheld, because it fails and a conformance run must not pass it. But it carries a KnownDeviation, which is the opposite call from @numeric-coercion next to it, and the difference is where the rule lives: numeric coercion is Appendix F borrowing flagd's ADR with no specification behind it, whereas codeDefaultFlag is normative OFREP. A bare omission would read as the same kind of choice, and this is not a choice. Untracked for now; delete both once handleResolved honours a value-less success. Nothing is owed upstream. The four disabled-* flags are served by the image already pinned, which the Compose header now records so the failures are not mistaken for another testbed gap. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
Import-path and coordinate churn only: the artifact is dev.openfeature.contrib.tools:tck, the version range starts at 0.1.0, and the four imports come from dev.openfeature.contrib.tools.tck. No behavioural change, and no change to what is declared or withheld. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
…tainers
Two fixes to this module's POM, both of which were latent.
testExclusions was missing. The parent POM feeds <exclude>${testExclusions}</exclude>
to Surefire but defines no default, so a module that wants the Docker gate has
to declare the property and a module that forgets one runs a Docker-dependent
suite in every job. providers/flagd declares it; this module did not, so
OfrepTckTest started a Compose stack during plain `mvn verify` and ended
"Tests run: 56, Failures: 2" on the recorded testbed gaps. So the ofrep PR was
red in CI while the flagd PR was green for the opposite reason - one ran a suite
it should gate, the other gated a suite nobody ran. Adding the exclusion makes
the policy uniform: `mvn -Pcodequality -pl providers/ofrep -am verify` is now
BUILD SUCCESS with 18 tests and no Compose stack.
Testcontainers is now declared here. The TCK moved it to provided/optional so
that the majority of adopters, which have no backend and never load
ContainerizedProviderTckTest, stop resolving it; a containerised adopter brings
its own instead. A previous pass asserted that both adoptions in this repository
already did - that was true of providers/flagd and false of this module, which
was relying on the transitive edge. Stated plainly because the assertion was
wrong, not because the fix is interesting.
Verified with the suite actually run: 56 scenarios, 38 passed, 16 skipped for
withheld capabilities, 2 failed on the flags flagd-testbed v3.8.0 does not
serve - unchanged, so the dependency now resolves from here rather than through
the TCK.
Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
The other half of adding testExclusions. An exclusion nobody writes down is indistinguishable from an oversight, and that is not a hypothetical here: this module's exclusion was missing precisely because nothing said the convention existed. So the README states the policy and its reasoning - Docker-gated, excluded from every job that exists, run by hand by a maintainer before merging a change to resolution or error behaviour, with the result quoted in the pull request. A scheduled or path-filtered workflow was considered and declined. It also gives the command and says which failures are expected, so a reader can tell a regression from the recorded testbed gaps, and notes that several capabilities are withheld because OFREP puts the decision on the server rather than in the provider - a fact about the protocol, not a defect. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
Appendix F now carries "Running the suite in CI", promoted there because the
same reasoning restated in four adoption READMEs is where it drifted. So this
section keeps the mechanism and the local record -- that this module had no
testExclusions at all until it was added, which is the appendix's
"exclusion nobody wrote down" -- and links to the appendix for the argument
rather than paraphrasing it.
Also states that no profile in this module touches the property, resolved rather
than read:
mvn -Pe2e -pl providers/ofrep help:evaluate -Dexpression=testExclusions
-> **/e2e/*.java
mvn -pl providers/ofrep help:evaluate -Dexpression=testExclusions
-> **/e2e/*.java
Unlike providers/flagd, which needs an e2e profile for its legacy suites and
therefore narrows the pattern instead of clearing it.
Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
This suite declares everything declarable except nine capabilities, so it picked @standard-reasons up by default the moment the TCK gained it. Measured before it was written down. Eight of reason.feature's nine scenarios run and pass: STATIC for the four rule-less flags, ERROR beside FLAG_NOT_FOUND and TYPE_MISMATCH, and -- because @targeting is declared here -- TARGETING_MATCH and DEFAULT either side of targeting-key-flag's rule. The ninth carries @disabled-flags as well and is skipped for that omission, which is the right outcome rather than a second report of the same gap: what this provider does wrong with a value-less success is already stated once, in the withheld capability and its KnownDeviation, and a reason it never reaches is not more evidence of it. So the tag means "the standard vocabulary, over the responses this provider actually completes", and a reader sees the withheld @disabled-flags beside it and can tell which scenario went unasked. A clean run is 65 scenarios, 46 passing, 17 skipped and 2 failing, up from 56, 38 and 16. The two failures are the same testbed gaps as before. Also records something this pass measured rather than introduced: the suite is intermittently flaky. About half of the runs carry one or two extra failures where an evaluation comes back as the code default, or as FLAG_NOT_FOUND where TYPE_MISMATCH was expected, or with reason ERROR where a resolution was expected. The victim moves between errors.feature, evaluation.feature and reason.feature, so it is not a property of any assertion. Eight runs were measured, five at this revision and three at ccdb8879, and the old pin produced a seven-failure run and a two-failure run from the same tree -- so this predates the reason scenarios and is not caused by them. That is the flagd-testbed readiness window of open-feature/flagd-testbed#394 reaching a provider that holds nothing between calls, so every evaluation races the stack afresh. Recorded in the class javadoc and the README with an explicit instruction not to cover it with a settle after control calls, because a suite that sleeps instead of holding the control API to its promise stops being able to detect when the promise breaks. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
This suite's declarableExcept list is long, and every name in it is something this provider genuinely cannot do -- no lifecycle to observe, no events, no connection to lose, a backend that decides so the caller's default never leaves the process, a deserialiser that keeps the numeric types strictly apart. @large-integers was the odd one out: it said nothing about OFREP at all, only that Client.getIntegerDetails is a 32-bit Integer. The TCK refuses it centrally now, so it is gone from the list and from the paragraph that explained it. That matters more here than in the flagd suite, because here it was one name among nine and a reader had no way to tell which of the nine described the provider. Now all of them do. Measured on the pinned testbed image: 65 scenarios, 46 passed, 17 skipped, 2 failed -- a clean run, identical to the previous pass, since this changes the reason for a skip rather than the count. The stream now carries four "provider does not declare capability" reasons and one "the Java SDK cannot express capability LARGE_INTEGERS", which is the distinction a reader of the report needs. No KnownDeviation changes: the existing DISABLED_FLAGS entry is unaffected, and a capability no Java provider can be asked was never a deviation to record. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
…ide it Two capability omissions in this suite are argued in the javadoc, and Appendix F's declaring rules have since drawn lines through both arguments. @numeric-coercion is withheld because the provider does not coerce -- it type-checks in both directions -- and no requirement says it must, which is the question that comes first and settles this one. The paragraph that follows also observed that declaring the tag would run a scenario the pinned testbed cannot serve. That is now explicitly not a reason: a scenario failing for a missing fixture is not a provider defect, so it argues for nothing. It stays as a note about what such a run would look like, marked as one. The appendix's scenario-level rule does not reach this omission either, and the javadoc says why rather than leaving a reader to wonder: that rule is about whether a question is askable, and all three of these are -- what it does not decide is whether an answer is owed. @disabled-flags is the opposite case and the javadoc now says so plainly. The provider receives the codeDefaultFlag response, parses its reason and answers with the wrong error code, so it does attempt the behaviour, and the settled guidance prefers declaring the tag and letting the four rows fail with this same deviation beside them. That is a change of results rather than of prose and is not made here; recording it is what stops it being lost. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
The same shape providers/flagd just gained, and for the same reason Appendix F gives: a conformance run carries failures by design wherever OfrepTckTest declares a knownDeviation, so a signal it shares with a suite that is expected green ends with somebody silencing the informative half. The `tck` profile clears this module's **/e2e/*.java exclusion and narrows Surefire's includes to **/e2e/*TckTest.java in the same breath, so `mvn -Ptck -pl providers/ofrep test` runs OfrepTckTest and nothing else - 65 scenarios, 46 passing, 17 skipped and 2 failing, which is the clean run the README already documents. This module had no profiles at all, so the README's claim that no profile touched testExclusions was true and now is not; it names the one that does. Nothing activates `tck` in CI. The README says not to add `-am` to that run, and why: it pulls tools/tck into the reactor and runs its 246 tests first, so a failure there comes out as a `-Ptck` failure. A one-off `install` is what `-am` was there for. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
The suite lived in an `e2e` package it was the only member of, and every selector reached it by filename: `**/e2e/*.java` to exclude it, `**/e2e/*TckTest.java` to run it. It now lives in `src/test/java/.../ofrep/tck/` and the selectors name that directory. The `e2e` package is gone -- there was never an end-to-end suite in this module for the adoption to sit beside, which is the clearest form of the argument: it was filed under a category it was not. The class drops what the directory now says. `OfrepTckTest` in package ...ofrep.tck said "tck" twice; it is `OfrepTest`. The `*Test` suffix stays, because Surefire's default includes need it -- that is a different thing from the selector being removed. The name a run is filed under is unchanged: configuration() strips the JUnit suffix either way, so it was and remains "ofrep". Both halves of the `tck` profile are still needed, for the reason they always were: the include alone leaves the exclusion in force and runs nothing, and dropping the exclusion alone runs this module's unit tests alongside the suite. testExclusions is still a Surefire and not a compiler exclusion, so the suite still compiles in the default build. The tally does not move: 65 scenarios, 46 passed, 17 skipped, 2 failed. Same scenarios, same results, a different directory. The profile comment documenting the run had kept `-am` on it, which the README already warns against; it says the command that works now. Signed-off-by: Simon Schrottner <simon.schrottner@flagsmith.com>
aepfli
force-pushed
the
feat/provider-tck-flagd
branch
from
September 13, 2026 21:04
1e3c1dc to
be8ed17
Compare
aepfli
force-pushed
the
feat/provider-tck-ofrep
branch
from
September 13, 2026 21:04
b001ced to
a34065c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The OFREP provider under the conformance suite. Three files: one test class, a Compose file, and the
test dependency in
providers/ofrep/pom.xml.No new infrastructure. The same
flagd-testbedimage the flagd suites use, exposing 8016 (OFREP)and 8080 (launchpad), no pinned host ports.
providers/ofrep'se2epackage is gone — the conformance suite had been its only member. Thatis the nesting argument in its clearest form: the adoption was filed under a category the module did
not have. What remains is
…/ofrep/tck/OfrepTest.java, selected by**/tck/*.javain both the defaultbuild's exclusions and the
tckprofile's includes.What it declares
Declared:
@object,@variants,@targeting,@standard-reasons.Withheld:
@lifecycle,@reinitialization,@events,@stale,@configuration-change,@unavailable,@disabled-flags,@numeric-coercion. (@large-integersis refused by theimplementation for every Java provider — a 32-bit
Integeraccessor — rather than withheld here.)Six of those share one root cause:
OfrepProvideris a bareFeatureProviderwith no lifecycle ofits own. It holds no state, opens no stream, runs no poll loop and does not override
initialize()—every evaluation is a fresh, independent HTTP POST. There is nothing in it that could observe a
backend transition, let alone report one. Declaring
@lifecyclewould make the readiness scenariopass vacuously: the
PROVIDER_READYa client sees is the SDK's, emitted identically forNoOpProviderand for a provider aimed at a dead port.Two are more interesting, and both were measured rather than read off the source:
@numeric-coercionis withheld and carries no deviation. Its three scenarios test all threecases and a provider declaring the tag must pass all three; this one passes one, and gets it right
for the wrong reason —
0.5as an integer is rejected because it is aDouble, not because it isfractional. Both lossless cases fail on the same exact-instance check. No deviation accompanies it
deliberately: Appendix F is explicit that this is the one capability the specification does not
define — OpenFeature has a single
numbertype, the rule is borrowed from flagd's ADR, and "aprovider that behaves differently is not violating the specification".
@disabled-flagsis withheld and does carry a deviation, which is the opposite call, and thedifference is where the rule lives. flagd's OFREP endpoint answers a disabled flag with
200and{"key":…,"reason":"DISABLED"}and novaluemember. That is well-formed OFREP — thecodeDefaultFlagmember schema, described as "A flag evaluation that defers to the code defaultvalue … The provider must use the code default value when processing this response" — and
handleResolvedreaches itsresponseValue == nullbranch and reportsFLAG_NOT_FOUND, discardingthe
reasonit parsed one field earlier. So aMUSTin the protocol this provider implements isunmet, and the gap is wider than the four rows that found it: every
codeDefaultFlagresponse isreported to the application as a failure on an evaluation that succeeded.
That last one is the declaration on this branch the settled guidance would shape differently, and it
is recorded rather than quietly left.
KnownDeviationprefers declared-and-failing and confineswithheld-and-skipped to a provider that cannot attempt the behaviour at all — and this provider does
attempt it. Flagging it explicitly because it is the kind of call a reviewer should get to make.
Running it, and the tally
A clean run is 65 scenarios: 46 pass, 17 skip, 2 fail. Both failures are the
large-integer-flagfixture gap that flagd-testbed#392 fills —
the mandatory precision scenario and the
max-int32@variantsrow — not provider defects, andneither gets a deviation entry.
Worth filing against the Java OFREP provider
initialize()means a misconfigured provider reportsREADY. Point it at a bogusbaseUrland every evaluation silently returns the code default with
GENERAL, while the client status sayseverything is fine. One round trip in
initialize()would fix it and unlock both@unavailablescenarios.
FLAG_NOT_FOUND— thecodeDefaultFlaggap above. This isthe one with a filed deviation against it.
EventProvider. OFREP's bulk-evaluation endpoint with ETag support is designed forexactly this; polling it would unlock
@events,@staleand@configuration-change.ParseErrorescapes the provider's own error mapping.Resolver.resolvecatches onlyGeneralError, butOfrepApithrowsParseErroronJsonProcessingException, and the two aresiblings under
OpenFeatureError. The SDK's blanket catch means an application still gets detailsrather than a throw, so the suite does not catch this — but the provider reports
PARSE_ERRORwhereits own design intends
GENERAL.nextAllowedRequestTime, after which every flag's evaluation throwsGeneralErroruntil thedeadline — including flags that were never rate-limited.
OfrepResponse.getMetadata()callsImmutableMap.copyOfon afield that stays
nullwhen the JSON omitsmetadata. Every live path launders the object throughResolution, whose constructor reads the field directly, so it is safe today — but a direct use ofa deserialised
OfrepResponsewould NPE.