GenericAdder - Parameter "regex_fields" cannot be set

[niklas-kunz]

I am trying to replicate the following generic adder configuration from an example config i was given:

---
filter: "..."
regex_fields:
  - data_stream.dataset

generic_adder:
  add:
    event.kind: "event"

description: "..."
---

However, the parameter "regex_field" cannot be configured. Is the example config above out-of-date or is the parameter missing?

[ekneg54]

could you please specify "cannot be configured"?
the rule should be fine and the regex_field capability should work. we use it.

do you have any error trace for us?

[niklas-kunz]

We are auto-generating an overview of all available configuration parameters that we are then provide via an API. In this overview, the parameter "regex_field" is missing:

{
  "fields": [
    {
      "fieldId": "specific_rules",
      "name": "specificRules",
      "type": "Array<string>",
      "typeClass": "typing.List[str]",
      "default": null,
      "required": false,
      "description": "List of rule locations to load rules from.\n    In addition to paths to file directories it is possible to retrieve rules from a URI.\n    For valid URI formats see :ref:`getters`.\n    "
    },
    {
      "fieldId": "generic_rules",
      "name": "genericRules",
      "type": "Array<string>",
      "typeClass": "typing.List[str]",
      "default": null,
      "required": false,
      "description": "List of rule locations to load rules from.\n    In addition to paths to file directories it is possible to retrieve rules from a URI.\n    For valid URI formats see :ref:`getters`.\n    "
    },
    {
      "fieldId": "apply_multiple_times",
      "name": "applyMultipleTimes",
      "type": "boolean | null",
      "typeClass": "typing.Optional[bool]",
      "default": "False",
      "required": false,
      "description": "Set if the processor should be applied multiple times. This enables further processing\n    of an output with the same processor."
    },
    {
      "fieldId": "sql_config",
      "name": "sqlConfig",
      "type": "object | null",
      "typeClass": "typing.Optional[dict]",
      "default": null,
      "required": false,
      "description": "\n    Configuration of the connection to a MySQL database and settings on how to add data from\n    the database.\n    This field is optional. The database feature will not be used if `sql_config` is omitted.\n    Has following subfields:\n\n    - `user` - The user to use when connecting to the MySQL database.\n    - `password` - The password to use when connecting to the MySQL database.\n    - `host` - The host to use when connecting to the MySQL database.\n    - `database` - The database name to use when connecting to the MySQL database.\n    - `table` - The table name to use when connecting to the MySQL database.\n    - `target_column` - The name of the column whose values are being matched against a value\n      from an event.\n      If a value matches, the remaining values of the row with the match are being added to\n      the event.\n    - `add_target_column` - Determines if the target column itself will be added to the event.\n      This is set to false per default.\n    - `timer` - Period how long to wait (in seconds) before the database table is being checked\n      for changes.\n      If there is a change, the table is reloaded by Logprep.\n    - `file_lock_path` - Path to a file lock used by the adder when updating the SQL table\n      (default: ./sql_update.lock).\n    - `db_file_path` - Path to a file used to store the SQL table obtained by the generic adder\n      (default: ./sql_db_table.json).\n    "
    }
  ]
}

I suppose that this is because the parameter is missing in processor.py/rule.py and/or init=False.

[ekneg54]

what you show are all processor config parameters.
but the "regex_fields" parameter ist a rule parameter.

it is defined here:

Logprep/logprep/processor/base/rule.py

Line 159 in a8511be

regex_fields: list = field(validator=validators.instance_of(list), factory=list)

[niklas-kunz]

Thanks for the feedback. We will look into it and the issue can be closed for now.