feat: add optional queryInterval to throttle Microsoft Graph queries#118
Merged
Conversation
Add a `queryInterval` input field (a Go duration string, e.g. "10m") that lets the function skip querying Microsoft Graph until the interval has elapsed since its last successful query, independent of how often the Composition reconciles. This helps avoid Graph API throttling. Because a composition function cannot mutate XR metadata, the last-query timestamp is persisted as a `lastQueryTime` element appended to the result stored at the status target and read back on subsequent reconciles. The current time is sourced from the existing TimerInterface to keep the logic deterministic under test. Interval limiting is effective in Composition mode with a "status." target only; invalid durations are surfaced as a fatal result. Also regenerate the deepcopy and packaged CRD (which additionally picks up the previously missing failOnEmpty field). Signed-off-by: Yury Tsarev <yury@upbound.io>
Document how to exercise queryInterval with `crossplane render`: - Add a "Query Interval (Throttling)" section to example/README.md with runnable render commands for both the run+embed and skip scenarios. - Add example/xr-with-last-query-time.yaml, an observed XR carrying a lastQueryTime so the skip path can be demonstrated deterministically. - Use `-r` (not `-rc`) and add a macOS note: `-c`/--include-context hangs under crossplane CLI v2.x + Docker Desktop because the context function is reached over a unix socket bind-mounted into the render container. This is crossplane/cli#161, fixed on main by #163 but not yet released. Signed-off-by: Yury Tsarev <yury@upbound.io>
Add example/e2e/ with a runnable Composition-mode end-to-end setup for the queryInterval feature: a Function install manifest, a Composition using queryInterval: "2m" targeting status.validatedUsers, an XR, and a short README covering run/verify steps. Reuses example/definition.yaml (XRD) and the local azure-creds secret (kept out of git). Validated on kind + Crossplane 2.3.3 with function-msgraph:v0.7.0-rc3: first run queries and embeds lastQueryTime, reconciles within the interval skip (FunctionSkip/IntervalLimit), and the query refreshes only after the interval elapses (queries=2 across ~16 reconciles). Signed-off-by: Yury Tsarev <yury@upbound.io>
jakubramut
reviewed
Jul 16, 2026
…ined skipQueryWhenTargetHasData takes precedence once the target has data, which silently suppresses the queryInterval refresh. Surface this at runtime with a non-fatal warning instead of relying on the README alone, so a conflicting configuration is visible without breaking existing setups. Adds the warning in validateAndPrepareInput, a unit test, and a README note. Signed-off-by: Yury Tsarev <yury@upbound.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an optional
queryIntervalinput field (a Go duration string, e.g."10m"). When set, the function records the timestamp of its last successful query and skips querying Microsoft Graph again until the interval has elapsed — independent of how often the Composition reconciles. This mitigates Microsoft Graph API throttling.How
queryInterval *stringinput, parsed withtime.ParseDuration(invalid/non-positive → fatal result).lastQueryTimeelement in the result written to thestatus.target. A composition function can't mutate XR metadata, so the status target is the only durable, function-writable state to read back on the next reconcile.FunctionSkip/IntervalLimitcondition. The current time comes from the existingTimerInterface, keeping the logic deterministic under test.status.targets (context isn't persisted across reconciles; Operation mode writes annotations, not status).failOnEmptyfield).Testing
Unit
Added table-driven cases: skip within interval, refresh after elapsed, first-run embed, invalid duration.
go test ./...,go vet, andgolangci-lint(gocyclo ≤ 10) all pass.Local render (
crossplane render)lastQueryTimeappended tostatus.validatedUsers.FunctionSkip/IntervalLimit, status preserved.Cluster e2e (kind + Crossplane 2.3.3,
function-msgraph:v0.7.0-rc3)Composition with
queryInterval: "2m"→status.validatedUsers:lastQueryTimeFunctionSkip/IntervalLimit, status unchangedlastQueryTimeadvancedDocs & examples
example/user-validation-example-query-interval.yaml,example/xr-with-last-query-time.yaml.example/e2e/— runnable cluster e2e manifests + README.