refactor(engine): feed engine with signatures events

[AlonZivony]

Feed the signatures engine with its own output instead of feeding it to the output channel.
This allows signatures to use other signatures as their selector.

1. Explain what the PR does

cfa2e5d refactor(engine): feed engine with signatures events

fix #3680

2. Explain how to test it

3. Other comments

[AlonZivony]

I am debating if we should maybe change the Source of the selector to be "engine" or something similar.
This will allow the to easily distinguish between signatures and normal events in other signatures, and allow a signature to use all signature events in the selector (like we do in the policy with tags).

[rafaeldtinoco]

I like the idea of deriving the signatures (I believe the proctree example had a good "usecase" or, at least, an idea that could be used in that regards).

I think the PR needs improvements:

1. You need to better document existing and new logic and functions (please watch out for formatting/syntax, capitalize sentences, keep good line height, etc).

2. I'm afraid of the loop reintroduction. That could lead us to infinite loops if the mask isn't cleared by the signature engine in some occasion. Could you at least document (in a comment ?) what condition makes it safe ? Maybe we should have a counter for the event (can't rely in the event pointer in a LRU map because every time the event comes it has a new address since it was io.copied()).

3. I think we need to fix the "Args" part. When reintroducing the event, the FindingToEvent->NewEvent->getArguments logic nests the Args->TriggeredBy over and over and over. This is actually a good question, @NDStrahilevitz, you asked me about multiple derivations from a single event. Lets assume we will allow that to happen (for both events and signature events), how should "Args" and "triggeredBy" look like in that case ?

{
  "DifferencesInArgs": {
    "Event1": {
      "Args": [
        {
          "Name": "triggeredBy",
          "Value": {
            "args": [
              {
                "Name": "file_path",
                "Type": "const char*",
                "Value": "/home/rafaeldtinoco/work/projects/tracee/file_modification.txt"
              },
              {
                "Name": "dev",
                "Type": "dev_t",
                "Value": 271581188
              },
              {
                "Name": "inode",
                "Type": "unsigned long",
                "Value": 17214308
              },
              {
                "Name": "old_ctime",
                "Type": "unsigned long",
                "Value": 1699921967398957690
              },
              {
                "Name": "new_ctime",
                "Type": "unsigned long",
                "Value": 1699921967402291023
              }
            ],
            "id": 774,
            "name": "file_modification",
            "returnValue": 0
          }
        }
      ]
    },
    "Event2": {
      "Args": [
        {
          "Name": "triggeredBy",
          "Value": {
            "args": [
              {
                "Name": "triggeredBy",
                "Value": {
                  "args": [
                    {
                      "Name": "file_path",
                      "Type": "const char*",
                      "Value": "/home/rafaeldtinoco/work/projects/tracee/file_modification.txt"
                    },
                    {
                      "Name": "dev",
                      "Type": "dev_t",
                      "Value": 271581188
                    },
                    {
                      "Name": "inode",
                      "Type": "unsigned long",
                      "Value": 17214308
                    },
                    {
                      "Name": "old_ctime",
                      "Type": "unsigned long",
                      "Value": 1699921967398957690
                    },
                    {
                      "Name": "new_ctime",
                      "Type": "unsigned long",
                      "Value": 1699921967402291023
                    }
                  ],
                  "id": 774,
                  "name": "file_modification",
                  "returnValue": 0
                }
              }
            ],
            "id": 6001,
            "name": "FILE_MODIFICATION",
            "returnValue": 0
          }
        }
      ]
    }
  }
}

[AlonZivony]

1. You need to better document existing and new logic and functions (please watch out for formatting/syntax, capitalize sentences, keep good line height, etc).

Sure, though I feel it is already well documented in the code. You want in the documentation man pages?

2. I'm afraid of the loop reintroduction. That could lead us to infinite loops if the mask isn't cleared by the signature engine in some occasion. Could you at least document (in a comment ?) what condition makes it safe ? Maybe we should have a counter for the event (can't rely in the event pointer in a LRU map because every time the event comes it has a new address since it was io.copied()).

I don't really see a reason to be concerned here.
The only case this might be problematic is if the signatures will create a Finding which is of the same event type of the input event, which doesn't makes sense in the current way signatures work.
If a user creates two signatures which are derived from one another, then this is just a very bad logic from his part and its not our responsebillity (like endless recursion is not prevented by the compiler).
However, if you think this is necessary, I will add a warning in the documentation about creating mutual derivation of signatures or create a signature that its Finding is of a normal event type.

3. I think we need to fix the "Args" part. When reintroducing the event, the FindingToEvent->NewEvent->getArguments logic nests the Args->TriggeredBy over and over and over. This is actually a good question, @NDStrahilevitz, you asked me about multiple derivations from a single event. Lets assume we will allow that to happen (for both events and signature events), how should "Args" and "triggeredBy" look like in that case ?

I don't see a problem with nested derivedBy argument, but I agree this is open to discussion. @NDStrahilevitz tell us what you think

[NDStrahilevitz]

Not -1ing because @rafaeldtinoco raised the concerns relevant for it already.

I'll answer to the points:

1. Documentation - 100% required for this kind of change. Inside the code it didn't seem bad at all to me, but I think that the actual docs should be updated on this.
2. I tend to agree with @AlonZivony on the risk of an infinite loop, but just to be sure we should test this by loading two signatures, one regular one derived, add a print in the engine post selector stage, and trigger the initial signature. Then we can see if an infinite loop is generated anyway.
3. While I think one nested triggeredBy is ok, i'm concerned with any more than that. I'm not sure it's worthwhile for us to limit it though, so it should be left to the user's concern when writing signatures, and documented as a potential "risk" factor of the feature.

[AlonZivony]

@rafaeldtinoco @NDStrahilevitz
After going over the current documentation, it seems that there is no documentation page that go into the details of creating a signature.
It means that the fact that you can't use signatures in the events selector until now was not documented anywhere.
Unless you want me to write such detailed documentation myself (which either way shouldn't be part of this PR) then I don't think further documentation is required.
Feel free to reply to this if I missed something or you have an idea of a documentation you do want me to write.

[rafaeldtinoco]

@rafaeldtinoco @NDStrahilevitz After going over the current documentation, it seems that there is no documentation page that go into the details of creating a signature.

We have:

https://aquasecurity.github.io/tracee/dev/docs/events/custom/overview/
https://aquasecurity.github.io/tracee/dev/docs/events/custom/golang/

but they are very small. The fact that golang signatures aren't good end-user experience is definitely what made us not to extend them (as we had the rego x go-cel x golang signatures discussions).

It means that the fact that you can't use signatures in the events selector until now was not documented anywhere.

Sure, and this was only "a thing" after the "everything as event" feature was done (very recently) anyway.

Unless you want me to write such detailed documentation myself (which either way shouldn't be part of this PR) then I don't think further documentation is required.

Nah, its alright from myside.

[rafaeldtinoco]

LGTM. Nadav, Alon and I discussed this offline and agreed that the recursive triggeredBy is ok for now, all other items were discussed and already taken care.