Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

9.0: Correctly set Correlation and Causation IDs #3887

Open
skurfuerst opened this issue Sep 9, 2022 · 6 comments
Open

9.0: Correctly set Correlation and Causation IDs #3887

skurfuerst opened this issue Sep 9, 2022 · 6 comments
Labels

Comments

@skurfuerst
Copy link
Member

skurfuerst commented Sep 9, 2022

  • from 1 Command, multiple Events occur (Create Node with tethered child nodes)
    • Causation id (seems to be already impl.; not yet tested)
  • Publishing of nodes
    • Correlation ID for full publish process for ALL involved events.

Causation ID == "Command ID" (we don't store it, just conceptually)


PROBLEMS / concepts:

  • Publishing (applying events to other stream)
  • Rebase (applying COMMANDS to other stream) - maybe later "applying events to other stream, but they can fail"
  • Node with tethered children (multiple events from one command)

Intuition:

  • Causation ID should not change once set.
@bwaidelich
Copy link
Member

Our current idea, as discussed:

  • initiatingUserId, initiatingTimestamp and causationId are copied from initial events (Note: currently from the first event of a batch, which has the command attached to it in the metadata))
  • correlationId is always set/overridden on publishing/rebasing (= workspace related operations)

Example

Authenticated User "user1"

Create Node "home"

with tethered node "home/main"

  • NodeAggregateWasCreated

    • stream: "cs-user1-a"
    • id: "e1"
    • initiatingUserId: "user1"
    • initiatingTimestamp "t1"
    • causationId "causationId1" (generated) (TODAY: not set)
    • correlationId: ~
  • NodeAggregateWasCreated "home/main" (tethered)

    • stream: "cs-user1-a"
    • id "e2"
    • initiatingUserId "user1"
    • initiatingTimestamp "t1'"
    • causationId "causationId1" (generated) (TODAY: "e1")
    • correlationId: ~

Authenticated User "user2"

Create node "service"

  • NodeAggregateWasCreated "service"
    • stream: "cs-user2-a"
    • id "e3"
    • initiatingUserId "user2"
    • initiatingTimestamp "t2"
    • causationId "causationId2" (generated)
    • correlationId: ~

Authenticated User "user3"

Publishing of "user2" changes

just imagined, this is not really possible – but it's relevant for review workspaces

  • no rebase necessary (TODAY: we always rebase(?))

  • => copying of events from "cs-user2-a" to "cs-live"

  • NodeAggregateWasCreated "service"

    • stream: "cs-live"
    • id "e4"
    • initiatingUserId "user2" (copied from the original event "e3")
    • initiatingTimestamp "t2" (copied from the original event "e3")
    • causationId "causationId2" (copied from the original event "e3")
    • correlationId: "correlationId1"
  • WorkspaceWasPublished

    • stream: "workspace-live"
    • id: "e5"
    • initiatingUserId "user3"
    • initiatingTimestamp "t3"
    • causationId (?)
    • correlationId: "correlationId1" (generated for this publishing process)

Authenticated User "user1"

Rebasing

  • From "cs-user1-a" to "cs-user1-b" (because base content stream ("live") has new events)

  • TODAY: Re-handle commands, future hopefully: Re-apply events (can fail)

  • NodeAggregateWasCreated "home"

    • stream: "cs-user1-b"
    • id "e6"
    • initiatingUserId "user1" (copied from event "e1")
    • initiatingTimestamp "t1" (copied from event "e1")
    • causationId "causationId1" (copied from event "e1")
    • correlationId: "correlationId2"
  • NodeAggregateWasCreated "home/main" (tethered)

    • stream: "cs-user1-a"
    • id "e7"
    • initiatingUserId "user1" (copied from event "e1" because the command is attached to "e1", FUTURE: copied from "e2")
    • initiatingTimestamp "t1'" (copied from event "e1" because the command is attached to "e1", FUTURE: copied from "e2")
    • causationId "causationId1" (copied from event "e1" because the command is attached to "e1", FUTURE: copied from "e2")
    • correlationId: "correlationId2"
  • WorkspaceWasRebased

    • stream: "workspace-user1"
    • id "e8"
    • initiatingUserId "user1" (NOT copied!)
    • initiatingTimestamp "t4"
    • causationId (?)
    • correlationId: "correlationId2" (generated for this rebasing process)

Publishing of "user1" changes to "live"

  • NodeAggregateWasCreated "home"

    • stream: "cs-live"
    • id "e9"
    • initiatingUserId "user1" (copied from event "e6")
    • initiatingTimestamp "t1" (copied from event "e6")
    • causationId "causationId1" (copied from event "e6")
    • correlationId: "correlationId3"
  • NodeAggregateWasCreated "home/main" (tethered)

    • stream: "cs-live"
    • id "e10"
    • initiatingUserId "user1" (copied from event "e6" because the command is attached to "e1", FUTURE: copied from "e7")
    • initiatingTimestamp "t1'" (copied from event "e6" because the command is attached to "e1", FUTURE: copied from "e7")
    • causationId "causationId1" (copied from event "e6" because the command is attached to "e1", FUTURE: copied from "e7")
    • correlationId: "correlationId3"
  • WorkspaceWasPublished

    • stream: "workspace-live"
    • id: "e11"
    • initiatingUserId "user1"
    • initiatingTimestamp "t5"
    • causationId (?)
    • correlationId: "correlationId3" (generated for this publishing process)

bwaidelich added a commit that referenced this issue Sep 9, 2022
Makes the `UserIdProvider` a concept of the core package so that
the user id can be determined "lazily".

In practice this means that we can use the CR instance for _reading_
already before the authenticated user can be determined.

When handling commands, an exception will be thrown if no user can
be determined at this point.

This also adjusts the event decoration in `\Neos\ContentRepository\Core\ContentRepository::handle()`
such that `initiatingUserId` and `initiatingTimestamp` is not overridden
once it is set (see #3887)
@mhsdesign mhsdesign added the 9.0 label Jun 11, 2023
@bwaidelich
Copy link
Member

After an issue @ahaeslich had today with lots of ContentStreamWasForked events without corresponding Workspace events (see Slack discussion) this missing feature gets more critical and made me think:
The causationId does not have to be a UUID but it could contain some pre/suffix explaining the corresponding interaction, for example:
rebaseWorkspace-62a2c5d06b834252b686e00ce6aa2548.

If it turns out to be too much work to introduce the complete causation/correlation handling for now, we should at least add some property to the ForkContentStream command (and the corresponding event), e.g.:

ForkContentStream::create(
    $rebasedContentStream,
    $baseWorkspace->currentContentStreamId,
    sprintf('Rebase workspace "%s" (content stream "%s"), $command->workspaceName->value, $command->contentStreamId->value),
)

@mhsdesign
Copy link
Member

Ah this issue also made it to our list of big questions. But i dont really understand what is being discussed here, and it troubles me as this topic is marked critical ^^

@bwaidelich
Copy link
Member

it troubles me as this topic is marked critical

Don't worry. This is critical for debugging the event log, but at least the last suggestion should be very easy to add and without any breaking changes

@bwaidelich
Copy link
Member

If it turns out to be too much work to introduce the complete causation/correlation handling for now,
we should at least add some property to the ForkContentStream command

As discussed, I created a separate high-prio issue (#4758) for that part and remove this one from the priority list

@bwaidelich
Copy link
Member

bwaidelich commented Apr 28, 2024

Reminder: We should also set some kind of "originalEventId" metadata to events published to other content streams in order to correlate those.

This will allow us to detect "merges" rather than new events, i.e. instead of

%%{init: { 'logLevel': 'debug', 'theme': 'base', 'gitGraph': {'showBranches': true, 'showCommitLabel':true,'mainBranchName': 'Live', 'parallelCommits': false, 'rotateCommitLabel': true}} }%%
      gitGraph LR:
        branch "content stream A"
        commit id:"Node A created"
        commit id:"Node A properties set"
        commit id:"content stream A closed"
        branch "content stream B"
        commit id:"Content A stream forked'"
        commit id:"Node A created'"
        commit id:"Node A properties set'"
        commit id: "content stream B closed"
        branch "content stream C"
        commit id:"content stream B forked"
        commit id:"Node A created''"
        commit id:"Node A properties set''"
        checkout Live
        commit id:"Node A created'''"
        commit id:"Node A properties set'''"
        checkout "content stream C"
        commit id:"Content stream C removed"
        checkout "content stream B"
        commit id:"content stream B removed"

we could produce sth like

%%{init: { 'logLevel': 'debug', 'theme': 'base', 'gitGraph': {'showBranches': true, 'showCommitLabel':true,'mainBranchName': 'Live', 'parallelCommits': false, 'rotateCommitLabel': true}} }%%
      gitGraph LR:
        checkout Live
        commit id:"created"
        branch "content stream A"
        commit id:"Node A created"
        commit id:"Node A properties set"
        commit id:"content stream A closed"
        branch "content stream B"
        commit id:"content stream A forked'"
        merge "content stream A"
        checkout "content stream B"
        commit id: "content stream B closed"
        branch "content stream C"
        commit id:"content stream B forked"
        merge "content stream B"
        checkout Live
        merge "content stream C"
        checkout "content stream C"
        commit id:"content stream C removed"
        checkout "content stream B"
        commit id:"content stream B removed"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Todo
Development

No branches or pull requests

3 participants