Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#428 - temporarily removing evict valid time ranges #438

Closed

Conversation

jarohen
Copy link
Member

@jarohen jarohen commented Nov 25, 2019

This PR removes the time ranges and boolean flags from the evict tx-op, until we can define a consistent semantics for how they should behave (see #428 for more details and rationale)

N.B.:

  • After this is released, there won't be any new evict ops with time ranges on Kafka logs, but there may still be existing events
  • With this in mind, we hard-stop the indexer if we encountered such an event during replay. The user can then manually intervene by setting the CRUX_EVICT_TIME_RANGES environment variable to EVICT_ALL, which tells the indexer to ignore the time ranges and evict the document for all valid times. (in fact, the error message points the user to this PR)
  • We do intend to replace this behaviour in the future, but only once we've nailed down exactly how it should behave.
  • Other evict docs + tests updated (and removed, where it didn't make sense to update the examples)

@jarohen jarohen self-assigned this Nov 25, 2019
@jarohen jarohen force-pushed the 428-removing-evict-valid-time-ranges branch from 0c9c622 to d66a37f Compare November 25, 2019 17:02
@jarohen jarohen changed the title #428 - removing evict valid time ranges #428 - temporarily removing evict valid time ranges Nov 25, 2019
@jarohen jarohen force-pushed the 428-removing-evict-valid-time-ranges branch from d66a37f to 4fadb5b Compare November 25, 2019 17:23
@@ -96,26 +96,34 @@
tx-id)
(c/->id-buffer new-v)]]}))

(defn tx-command-evict [indexer kv object-store snapshot tx-log [op k start-valid-time end-valid-time keep-latest? keep-earliest?] transact-time tx-id]
(def evict-time-ranges-env-var "CRUX_EVICT_TIME_RANGES")
(def ^:dynamic evict-all-on-legacy-time-ranges? (= (System/getenv evict-time-ranges-env-var) "EVICT_ALL"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is dynamic it should have a *evict...* name.

@jarohen jarohen force-pushed the 428-removing-evict-valid-time-ranges branch from 3d404a3 to 5a2e725 Compare November 27, 2019 08:29
@jarohen jarohen closed this Nov 27, 2019
@jarohen jarohen force-pushed the 428-removing-evict-valid-time-ranges branch from 5a2e725 to d6c0517 Compare November 27, 2019 08:31
@jarohen jarohen deleted the 428-removing-evict-valid-time-ranges branch November 27, 2019 08:42
@jarohen
Copy link
Member Author

jarohen commented Nov 27, 2019

merged

@jarohen jarohen added the 1.x label Apr 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants