Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Form renders extra fields that breaks validation #4088

Open
3 of 4 tasks
kevinehosford opened this issue Feb 12, 2024 · 1 comment
Open
3 of 4 tasks

Form renders extra fields that breaks validation #4088

kevinehosford opened this issue Feb 12, 2024 · 1 comment

Comments

@kevinehosford
Copy link

kevinehosford commented Feb 12, 2024

Prerequisites

What theme are you using?

core

Version

5.x

Current Behavior

Using the playground, I added my json schema which renders the form; however, it appears erroneous extra components are rendered and are included on submit which breaks validation.

Expected Behavior

Only the fields specified in the schema given the current form state should be shown or included in validation.

Steps To Reproduce

  1. In the playground
  2. Using this following schema, notice the extra field label shown at the bottom. If you manipulate the kind select and try to validate, notice the complaint about "additional properties" should not be included.
    additional-properties
    extra
{
  "description": "Eventflow destination configuration schema.",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "required": [
    "id",
    "kind",
    "name",
    "created",
    "modified",
    "state",
    "stateTrigger",
    "stateMessage",
    "properties"
  ],
  "properties": {
    "id": {
      "type": "string"
    },
    "kind": {
      "type": "string",
      "enum": [
        "amqp",
        "aws_cloudwatch_logs",
        "aws_kinesis_firehose",
        "aws_kinesis_streams",
        "aws_s3",
        "aws_sqs",
        "axiom",
        "clickhouse",
        "cloudamqp",
        "datadog_events",
        "datadog_logs",
        "elasticsearch",
        "http",
        "humio_logs",
        "influxdb_logs",
        "kafka",
        "loki",
        "mezmo",
        "nats",
        "new_relic",
        "papertrail",
        "pulsar",
        "redis",
        "sematext_logs",
        "splunk_hec_logs"
      ]
    },
    "name": {
      "type": "string"
    },
    "created": {
      "type": "string"
    },
    "modified": {
      "type": "string"
    },
    "state": {
      "type": "string",
      "enum": [
        "active",
        "disabled",
        "errored"
      ]
    },
    "stateTrigger": {
      "type": "string",
      "enum": [
        "user",
        "system"
      ]
    },
    "stateMessage": {
      "type": [
        "string",
        "null"
      ]
    }
  },
  "allOf": [
    {
      "title": "AMQP protocol destination",
      "if": {
        "properties": {
          "kind": {
            "const": "amqp"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/amqp"
      }
    },
    {
      "title": "AWS Cloudwatch logs destination",
      "if": {
        "properties": {
          "kind": {
            "const": "aws_cloudwatch_logs"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/aws_cloudwatch_logs"
      }
    },
    {
      "title": "AWS Kinesis Firehose destination",
      "if": {
        "properties": {
          "kind": {
            "const": "aws_kinesis_firehose"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/aws_kinesis_firehose"
      }
    },
    {
      "title": "AWS Kinesis Streams destination",
      "if": {
        "properties": {
          "kind": {
            "const": "aws_kinesis_streams"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/aws_kinesis_streams"
      }
    },
    {
      "title": "AWS S3 destination",
      "if": {
        "properties": {
          "kind": {
            "const": "aws_s3"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/aws_s3"
      }
    },
    {
      "title": "AWS SQS destination",
      "if": {
        "properties": {
          "kind": {
            "const": "aws_qs"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/aws_sqs"
      }
    },
    {
      "title": "Axiom dataset ingest destination",
      "if": {
        "properties": {
          "kind": {
            "const": "axiom"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/axiom"
      }
    },
    {
      "title": "Clickhouse database destination",
      "if": {
        "properties": {
          "kind": {
            "const": "clickhouse"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/clickhouse"
      }
    },
    {
      "title": "Datadog Events API destination",
      "if": {
        "properties": {
          "kind": {
            "const": "datadog_events"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/datadog_events"
      }
    },
    {
      "title": "Datadog Logs API destination",
      "if": {
        "properties": {
          "kind": {
            "const": "datadog_logs"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/datadog_logs"
      }
    },
    {
      "title": "Elasticsearch Bulk API destination",
      "if": {
        "properties": {
          "kind": {
            "const": "elasticsearch"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/elasticsearch"
      }
    },
    {
      "title": "Generic HTTP/REST endpoint destination",
      "if": {
        "properties": {
          "kind": {
            "const": "http"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/http"
      }
    },
    {
      "title": "Humio Logs destination",
      "if": {
        "properties": {
          "kind": {
            "const": "humio_logs"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/humio_logs"
      }
    },
    {
      "title": "Influx Database Logs destination",
      "if": {
        "properties": {
          "kind": {
            "const": "influxdb_logs"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/influxdb_logs"
      }
    },
    {
      "title": "Kafka destination",
      "if": {
        "properties": {
          "kind": {
            "const": "kafka"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/kafka"
      }
    },
    {
      "title": "Grafana Loki destination",
      "if": {
        "properties": {
          "kind": {
            "const": "loki"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/loki"
      }
    },
    {
      "title": "Mezmo ( LogDNA ) destination",
      "if": {
        "properties": {
          "kind": {
            "const": "mezmo"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/mezmo"
      }
    },
    {
      "title": "NATS destination",
      "if": {
        "properties": {
          "kind": {
            "const": "nats"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/nats"
      }
    },
    {
      "title": "New Relic destination",
      "if": {
        "properties": {
          "kind": {
            "const": "new_relic"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/new_relic"
      }
    },
    {
      "title": "Papertrail destination",
      "if": {
        "properties": {
          "kind": {
            "const": "papertrail"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/papertrail"
      }
    },
    {
      "title": "Apache Pulsar destination",
      "if": {
        "properties": {
          "kind": {
            "const": "pulsar"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/pulsar"
      }
    },
    {
      "title": "Redis destination",
      "if": {
        "properties": {
          "kind": {
            "const": "redis"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/redis"
      }
    },
    {
      "title": "Sematext Logs destination",
      "if": {
        "properties": {
          "kind": {
            "const": "sematext_logs"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/sematext_logs"
      }
    },
    {
      "title": "Splunk HEC Logs destination",
      "if": {
        "properties": {
          "kind": {
            "const": "splunk_hec_logs"
          }
        }
      },
      "then": {
        "$ref": "#/definitions/splunk_hec_logs"
      }
    }
  ],
  "definitions": {
    "amqp": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "properties": {
            "connection_string": {
              "type": "string",
              "description": "AMQP connection string in URL form following the scheme: amqp[s]://<username>:<password>@<host>:<port>/<path>"
            },
            "exchange": {
              "type": "string",
              "description": "Optional AMQP exchange name"
            }
          },
          "required": [
            "connection_string"
          ],
          "additionalProperties": false
        }
      }
    },
    "aws_cloudwatch_logs": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "properties": {
            "auth": {
              "$ref": "#/definitions/aws_auth",
              "description": "AWS Authentication"
            },
            "group_name": {
              "type": "string",
              "description": "Name of the destination log group"
            },
            "compression": {
              "type": "string",
              "enum": [
                "none",
                "gzip",
                "zstd"
              ]
            },
            "create_missing_stream": {
              "type": "boolean",
              "description": "Create the destination stream if it does not exist"
            },
            "stream_name": {
              "type": "string",
              "description": "Name of the destination stream"
            },
            "region": {
              "type": "string",
              "description": "AWS region"
            }
          },
          "required": [
            "group_name",
            "stream_name"
          ],
          "additionalProperties": false
        }
      }
    },
    "aws_kinesis_firehose": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "properties": {
            "auth": {
              "$ref": "#/definitions/aws_auth",
              "description": "AWS Authentication"
            },
            "stream_name": {
              "type": "string",
              "description": "Name of the destination delivery stream"
            },
            "compression": {
              "type": "string",
              "description": "Compression algorithm to use",
              "enum": [
                "none",
                "gzip",
                "zstd"
              ]
            },
            "region": {
              "type": "string",
              "description": "AWS region"
            }
          },
          "additionalProperties": false
        }
      }
    },
    "aws_kinesis_streams": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "properties": {
            "auth": {
              "$ref": "#/definitions/aws_auth",
              "description": "AWS Authentication"
            },
            "stream_name": {
              "type": "string",
              "description": "Name of the destination stream"
            },
            "compression": {
              "type": "string",
              "description": "Compression algorithm to use",
              "enum": [
                "none",
                "gzip",
                "zstd"
              ]
            },
            "create_missing_stream": {
              "type": "boolean",
              "description": "Create the destination stream if it does not exist"
            },
            "region": {
              "type": "string",
              "description": "AWS region"
            }
          },
          "required": [
            "stream_name"
          ],
          "additionalProperties": false
        }
      }
    },
    "aws_s3": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "properties": {
            "auth": {
              "$ref": "#/definitions/aws_auth",
              "description": "AWS Authentication"
            },
            "bucket": {
              "type": "string",
              "description": "Name of the destination bucket"
            },
            "compression": {
              "type": "string",
              "description": "Compression algorithm to use",
              "default": "gzip",
              "enum": [
                "none",
                "gzip",
                "zstd"
              ]
            },
            "filename_append_uuid": {
              "type": "boolean",
              "description": "Should a UUID v4 token be appended to the end of the object key",
              "default": false
            },
            "filename_extension": {
              "type": [
                "string",
                "null"
              ],
              "description": "Filename extension to use for the destination object key. This overrides the extension based on the 'compression' setting."
            },
            "filename_time_format": {
              "type": [
                "string",
                "null"
              ],
              "default": "%s",
              "description": "Timestamp format to use for the time component of the destination object key. This overrides the time format based on the 'compression' setting. When set to empty no prefix is used. When omitted a default format is used of the form date=2023-11-05/1234123412."
            },
            "key_prefix": {
              "type": [
                "string",
                "null"
              ],
              "description": "Prefix to apply for all the destination object keys. If directory creation is desired, it should end with a trailing slash '/'."
            },
            "acl": {
              "type": [
                "string",
                "null"
              ],
              "default": null,
              "enum": [
                "private",
                "public-read",
                "public-read-write",
                "authenticated-read",
                "aws-exec-read",
                "bucket-owner-read",
                "bucket-owner-full-control",
                "log-delivery-write"
              ],
              "description": "ACL to apply to the destination object"
            },
            "content_encoding": {
              "type": [
                "string",
                "null"
              ],
              "description": "Content encoding to apply to the destination object"
            },
            "content_type": {
              "type": [
                "string",
                "null"
              ],
              "description": "Content type to apply to the destination object"
            },
            "grant_full_control": {
              "type": [
                "string",
                "null"
              ],
              "description": "Grant full control to the specified AWS user"
            },
            "grant_read": {
              "type": [
                "string",
                "null"
              ],
              "description": "Grant read access to the specified AWS user"
            },
            "grant_read_acp": {
              "type": [
                "string",
                "null"
              ],
              "description": "Grant read access to the specified AWS user"
            },
            "grant_write_acp": {
              "type": [
                "string",
                "null"
              ],
              "description": "Grant write access to the specified AWS user"
            },
            "region": {
              "type": "string",
              "description": "AWS region"
            },
            "server_side_encryption": {
              "type": [
                "string",
                "null"
              ],
              "default": null,
              "enum": [
                "AES256",
                "aws:kms"
              ],
              "description": "Server side encryption algorithm to use"
            },
            "sse_kms_key_id": {
              "type": [
                "string",
                "null"
              ],
              "description": "KMS key ID to use for server side encryption"
            },
            "storage_class": {
              "type": [
                "string",
                "null"
              ],
              "default": "STANDARD",
              "enum": [
                "STANDARD",
                "REDUCED_REDUNDANCY",
                "STANDARD_IA",
                "ONEZONE_IA",
                "INTELLIGENT_TIERING",
                "GLACIER",
                "DEEP_ARCHIVE"
              ],
              "description": "Storage class to use for the destination object"
            },
            "tags": {
              "type": [
                "object",
                "null"
              ],
              "default": {},
              "additionalProperties": {
                "type": "string"
              },
              "description": "Tags to apply to the destination object"
            },
            "path": {
              "type": "string",
              "description": "Path prefix for the destination object"
            }
          },
          "required": [
            "bucket"
          ],
          "additionalProperties": false
        }
      }
    },
    "aws_sqs": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "properties": {
            "auth": {
              "$ref": "#/definitions/aws_auth",
              "description": "AWS Authentication"
            },
            "queue_url": {
              "type": "string",
              "description": "The AWS SQS queue URL"
            },
            "message_group_id": {
              "type": [
                "string",
                "null"
              ],
              "description": "Message group ID to use for FIFO queues"
            },
            "message_deduplication_id": {
              "type": [
                "string",
                "null"
              ],
              "description": "Message deduplication ID to use for FIFO queues"
            },
            "region": {
              "type": "string",
              "description": "AWS region"
            }
          },
          "required": [
            "queue_url"
          ],
          "additionalProperties": false
        }
      }
    },
    "axiom": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "properties": {
            "url": {
              "type": [
                "string",
                "null"
              ],
              "default": null,
              "description": "Axiom ingest API endpoint"
            },
            "dataset": {
              "type": "string",
              "description": "Name of the destination dataset"
            },
            "org_id": {
              "type": [
                "string",
                "null"
              ],
              "default": null,
              "description": "Axiom organization ID"
            },
            "compression": {
              "type": "string",
              "description": "Compression algorithm to use",
              "enum": [
                "none",
                "gzip",
                "zstd"
              ]
            },
            "token": {
              "type": "string",
              "description": "Axiom ingest API token"
            },
            "region": {
              "type": "string",
              "description": "AWS region"
            }
          },
          "required": [
            "dataset",
            "token"
          ],
          "additionalProperties": false
        }
      }
    },
    "clickhouse": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "properties": {
            "endpoint": {
              "type": "string",
              "description": "Clickhouse connection string in URL form following the scheme: http[s]://<host>:<port>"
            },
            "auth": {
              "$ref": "#/definitions/http_auth",
              "description": "HTTP Authentication"
            },
            "table": {
              "type": "string",
              "description": "Name of the destination table"
            },
            "skip_unknown_fields": {
              "type": "boolean",
              "description": "Allows clickhouse to skip unknown fields not present in table schema"
            },
            "compression": {
              "type": "string",
              "description": "Compression algorithm to use",
              "enum": [
                "none",
                "gzip",
                "lz4",
                "zlib",
                "zstd",
                "snappy"
              ]
            },
            "database": {
              "type": "string",
              "description": "Name of the destination database"
            },
            "date_time_best_effort": {
              "type": "boolean",
              "description": "Allows clickhouse to parse date time fields per RFC3339/ISO 8601"
            }
          },
          "required": [
            "endpoint",
            "table"
          ],
          "additionalProperties": false
        }
      }
    },
    "datadog_events": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "properties": {
            "default_api_key": {
              "type": "string",
              "description": "DataDog API key"
            },
            "endpoint": {
              "type": "string",
              "description": "DataDog connection string in URL form following the scheme: http[s]://<host>:<port>. If set overrides `site` property"
            },
            "site": {
              "type": "string",
              "description": "DataDog deployment site. If set overrides `endpoint` property"
            },
            "compression": {
              "type": "string",
              "description": "Compression algorithm to use",
              "default": "gzip",
              "enum": [
                "none",
                "gzip",
                "lz4",
                "zlib",
                "zstd",
                "snappy"
              ]
            },
            "region": {
              "type": "string",
              "description": "Datadog deployment region"
            }
          },
          "required": [
            "default_api_key"
          ],
          "additionalProperties": false
        }
      }
    },
    "datadog_logs": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "properties": {
            "default_api_key": {
              "type": "string",
              "description": "DataDog API key"
            },
            "endpoint": {
              "type": "string",
              "description": "DataDog connection string in URL form following the scheme: http[s]://<host>:<port>. If set overrides `site` property"
            },
            "site": {
              "type": "string",
              "description": "DataDog deployment site. If set overrides `endpoint` property"
            },
            "compression": {
              "type": "string",
              "description": "Compression algorithm to use",
              "default": "gzip",
              "enum": [
                "none",
                "gzip",
                "lz4",
                "zlib",
                "zstd",
                "snappy"
              ]
            },
            "region": {
              "type": "string",
              "description": "Datadog deployment region"
            }
          },
          "required": [
            "default_api_key"
          ],
          "additionalProperties": false
        }
      }
    },
    "elasticsearch": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "properties": {
            "endpoint": {
              "type": "string",
              "description": "Elasticsearch connection string in URL form following the scheme: http[s]://<host>:<port>"
            },
            "index": {
              "type": "string",
              "default": "from-axiom-%Y.%m.%d",
              "description": "Name of the destination index"
            },
            "api_version": {
              "type": "string",
              "description": "Elasticsearch API version",
              "enum": [
                "auto",
                "v6",
                "v7",
                "v8"
              ]
            }
          },
          "required": [
            "endpoint",
            "index"
          ],
          "additionalProperties": false
        }
      }
    },
    "http": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "properties": {
            "auth": {
              "$ref": "#/definitions/http_auth",
              "description": "HTTP Authentication"
            },
            "uri": {
              "type": "string",
              "description": "URL of the destination endpoint"
            },
            "headers": {
              "type": "object",
              "description": "HTTP headers to send with each request",
              "additionalProperties": {
                "type": "string"
              }
            },
            "method": {
              "type": "string",
              "description": "HTTP method to use",
              "enum": [
                "get",
                "head",
                "post",
                "put",
                "patch",
                "delete",
                "options",
                "trace"
              ]
            },
            "content_type": {
              "type": "string",
              "description": "Content type of the request body",
              "enum": [
                "application/json",
                "application/x-www-form-urlencoded",
                "text/plain"
              ]
            },
            "compression": {
              "type": "string",
              "description": "Compression algorithm to use",
              "enum": [
                "none",
                "gzip",
                "lz4",
                "zlib",
                "zstd",
                "snappy"
              ]
            },
            "body": {
              "type": "string",
              "description": "Request body"
            }
          },
          "required": [
            "uri"
          ],
          "additionalProperties": false
        }
      }
    },
    "humio_logs": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "description": "Humio API endpoint configuration",
          "properties": {
            "token": {
              "type": "string",
              "description": "Humio API token"
            },
            "endpoint": {
              "type": "string",
              "default": "https://cloud.humio.com",
              "description": "Humio API host endpoint in URL form following the scheme: http[s]://<host>:<port>"
            },
            "compression": {
              "type": "string",
              "default": "gzip",
              "description": "Compression algorithm to use",
              "enum": [
                "none",
                "gzip",
                "lz4",
                "zlib",
                "zstd",
                "snappy"
              ]
            },
            "host_key": {
              "type": "string",
              "default": "host",
              "description": "Name of the field to use as the host key, or defaulted to 'host_key' data if not present"
            },
            "index": {
              "type": "string",
              "default": null,
              "description": "Name of the destination repository used to create ingest token"
            },
            "indexed_fields": {
              "type": "array",
              "description": "List of fields to add to Humio's extra fields",
              "items": {
                "type": "string"
              }
            }
          },
          "required": [
            "token"
          ],
          "additionalProperties": false
        }
      }
    },
    "influxdb_logs": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "description": "InfluxDB API endpoint configuration",
          "properties": {
            "endpoint": {
              "type": "string",
              "description": "InfluxDB connection string in URL form following the scheme: http[s]://<host>:<port>"
            },
            "database": {
              "type": "string",
              "description": "Name of the destination database"
            },
            "org": {
              "type": "string",
              "description": "Name of the destination organization"
            },
            "token": {
              "type": "string",
              "description": "InfluxDB API token"
            },
            "bucket": {
              "type": "string",
              "description": "Name of the destination bucket"
            }
          },
          "required": [
            "endpoint"
          ],
          "additionalProperties": false
        }
      }
    },
    "kafka": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "description": "Kafka broker configuration",
          "properties": {
            "bootstrap_servers": {
              "type": "array",
              "description": "List of Kafka broker bootstrap servers",
              "items": {
                "type": "string"
              }
            },
            "topic": {
              "type": "string",
              "description": "Name of the destination topic"
            },
            "compression": {
              "type": "string",
              "description": "Compression algorithm to use",
              "enum": [
                "none",
                "gzip",
                "lz4",
                "zstd",
                "snappy"
              ]
            },
            "headers_key": {
              "type": [
                "string",
                "null"
              ],
              "description": "Name of the field to use as the headers key, or unset if not present"
            },
            "key_field": {
              "type": [
                "string",
                "null"
              ],
              "description": "Name of the field to use as the topic key, or unset if not present"
            },
            "librdkafka_options": {
              "type": "object",
              "description": "librdkafka configuration options",
              "default": {},
              "additionalProperties": {
                "type": "string"
              }
            },
            "sasl": {
              "$ref": "#/definitions/kafka_auth",
              "description": "Kafka Authentication"
            }
          },
          "required": [
            "bootstrap_servers",
            "topic"
          ],
          "additionalProperties": false
        }
      }
    },
    "loki": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "description": "Loki API endpoint configuration",
          "properties": {
            "endpoint": {
              "type": "string",
              "description": "Loki connection string in URL form following the scheme: http[s]://<host>:<port>"
            },
            "auth": {
              "$ref": "#/definitions/http_auth",
              "description": "HTTP authentication"
            },
            "out_of_order_action": {
              "type": "string",
              "description": "Action to take when an out of order log entry is received",
              "enum": [
                "drop",
                "rewrite_timestamp",
                "accept"
              ]
            },
            "compression": {
              "type": "string",
              "description": "Compression algorithm to use",
              "enum": [
                "none",
                "gzip",
                "zlib",
                "zstd"
              ]
            },
            "labels": {
              "type": "object",
              "description": "Labels to attach to each log entry",
              "additionalProperties": {
                "type": "string"
              }
            },
            "path": {
              "type": "string",
              "default": "/loki/api/v1/push",
              "description": "Path to the use for the Loki instance"
            },
            "remove_label_fields": {
              "type": "boolean",
              "description": "Remove the label fields when they are used as labels"
            },
            "remove_timestamp": {
              "type": "boolean",
              "description": "Remove the timestamp from each log entry but perserve for loki index purposes"
            },
            "tenant_id": {
              "type": "string",
              "description": "Loki tenant ID"
            }
          },
          "required": [
            "endpoint"
          ],
          "additionalProperties": false
        }
      }
    },
    "mezmo": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "description": "Mezmo API endpoint configuration",
          "properties": {
            "api_key": {
              "type": "string",
              "description": "Mezmo Ingestion API key"
            },
            "default_app": {
              "type": "string",
              "description": "Mezmo default app - set when there is no 'app'/'file' field in data record"
            },
            "default_env": {
              "type": "string",
              "description": "Mezmo default environment - set when there is no 'env' field in data record"
            },
            "endpoint": {
              "type": "string",
              "description": "Mezmo HTTP API endpoint"
            },
            "ip": {
              "type": "string",
              "description": "The optional IP address to attach to each batch of events"
            },
            "mac": {
              "type": "string",
              "description": "The optional MAC address to attach to each batch of events"
            },
            "tags": {
              "type": "array",
              "description": "Mezmo API tags",
              "items": {
                "type": "string"
              }
            },
            "hostname": {
              "type": "string",
              "description": "The hostname to attach to each batch of events"
            }
          },
          "required": [
            "api_key",
            "hostname"
          ],
          "additionalProperties": false
        }
      }
    },
    "nats": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "description": "NATS broker configuration",
          "properties": {
            "auth": {
              "$ref": "#/definitions/nats_auth",
              "description": "NATS Authentication"
            },
            "url": {
              "type": "string",
              "description": "NATS destination URL of the form https://docs.nats.io/using-nats/developer/connecting#nats-url"
            },
            "subject": {
              "type": "string",
              "description": "Name of the destination subject"
            },
            "connection_name": {
              "type": "string",
              "description": "Name of the connection"
            }
          },
          "required": [
            "url",
            "subject"
          ],
          "additionalProperties": false
        }
      }
    },
    "new_relic": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "description": "New Relic API endpoint configuration",
          "properties": {
            "account_id": {
              "type": "string",
              "description": "New Relic account ID"
            },
            "api": {
              "type": "string",
              "description": "New Relic API version",
              "enum": [
                "events",
                "logs",
                "metrics"
              ]
            },
            "license_key": {
              "type": "string",
              "description": "New Relic license key"
            },
            "region": {
              "type": "string",
              "description": "New Relic deployment region"
            },
            "compression": {
              "type": "string",
              "description": "Compression algorithm to use",
              "enum": [
                "none",
                "gzip",
                "lz4",
                "zlib",
                "zstd",
                "snappy"
              ]
            }
          },
          "required": [
            "account_id",
            "api",
            "license_key"
          ],
          "additionalProperties": false
        }
      }
    },
    "papertrail": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "description": "Papertrail API endpoint configuration",
          "properties": {
            "endpoint": {
              "type": "string",
              "description": "Papertrail connection string in URL form following the scheme: <host>:<port>"
            },
            "process": {
              "type": "string",
              "description": "Name of the sending process - for example, the name of the eventflow or another user defined label"
            }
          },
          "required": [
            "endpoint"
          ],
          "additionalProperties": false
        }
      }
    },
    "pulsar": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "description": "Pulsar broker configuration",
          "properties": {
            "endpoint": {
              "type": "string",
              "description": "Pulsar server endpoint"
            },
            "topic": {
              "type": "string",
              "description": "Name of the destination topic"
            },
            "compression": {
              "type": "string",
              "description": "Compression algorithm to use",
              "enum": [
                "none",
                "gzip",
                "lz4",
                "zlib",
                "zstd",
                "snappy"
              ]
            },
            "producer_name": {
              "type": "string",
              "description": "Name of the producer"
            },
            "partition_key_field": {
              "type": "string",
              "description": "Name of the field to use as the partition key, or unset if not present"
            }
          },
          "required": [
            "endpoint",
            "topic"
          ],
          "additionalProperties": false
        }
      }
    },
    "redis": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "description": "Redis broker configuration",
          "properties": {
            "endpoint": {
              "type": "string",
              "description": "Redis connection string in URL form following the scheme: redis[s]://<username>:<password>@<host>:<port>/<path>"
            },
            "data_type": {
              "type": "string",
              "enum": [
                "list",
                "channel"
              ],
              "description": "Redis data type to use: Can be either list or channel"
            },
            "key": {
              "type": "string",
              "description": "Name of the destination key"
            }
          },
          "required": [
            "endpoint",
            "data_type"
          ],
          "additionalProperties": false
        }
      }
    },
    "sematext_logs": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "description": "Sematext Logs API endpoint configuration",
          "properties": {
            "endpoint": {
              "type": "string",
              "description": "Sematext Logs API endpoint"
            },
            "token": {
              "type": "string",
              "description": "Sematext Logs application token"
            },
            "region": {
              "type": "string",
              "description": "Sematext deployment region string"
            }
          },
          "required": [
            "endpoint",
            "token"
          ],
          "additionalProperties": false
        }
      }
    },
    "splunk_hec_logs": {
      "type": "object",
      "properties": {
        "properties": {
          "type": "object",
          "description": "Splunk HEC API endpoint configuration",
          "properties": {
            "default_token": {
              "type": "string",
              "description": "Splunk HEC token"
            },
            "endpoint": {
              "type": "string",
              "description": "Splunk HEC API host endpoint"
            }
          },
          "required": [
            "default_token",
            "endpoint"
          ],
          "additionalProperties": false
        }
      }
    },
    "aws_auth": {
      "description": "AWS Authentication Strategy",
      "anyOf": [
        {
          "description": "Use an access key and secret pair.",
          "type": "object",
          "required": [
            "access_key_id",
            "secret_access_key"
          ],
          "properties": {
            "access_key_id": {
              "description": "AWS access key identifier.",
              "type": "string"
            },
            "assume_role": {
              "title": "AWS ARN of an IAM role to assume.",
              "description": "[iam_role]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html",
              "type": [
                "string",
                "null"
              ]
            },
            "external_id": {
              "title": "An optional unique external identifier in conjunction with role to assume.",
              "description": "[external_id]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html",
              "type": [
                "string",
                "null"
              ]
            },
            "region": {
              "title": "AWS region to send STS requests to.",
              "description": "If not set, this will default to the configured region\nfor the service itself.",
              "type": [
                "string",
                "null"
              ]
            },
            "secret_access_key": {
              "description": "AWS secret access key.",
              "type": "string"
            }
          }
        },
        {
          "title": "Authenticate using file based credentials.",
          "description": "Credentials file of the form: <https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html>.",
          "type": "object",
          "required": [
            "credentials_file"
          ],
          "properties": {
            "credentials_file": {
              "description": "The system path to the credentials file.",
              "type": "string"
            },
            "profile": {
              "title": "The AWS credentials profile to use.",
              "description": "Used to select AWS credentials profile from a provided credentials file.",
              "default": "default",
              "type": "string"
            }
          }
        },
        {
          "description": "Assume the given role ARN.",
          "type": "object",
          "required": [
            "assume_role"
          ],
          "properties": {
            "assume_role": {
              "title": "AWS ARN of an IAM role to assume.",
              "description": "[iam_role]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html",
              "type": "string"
            },
            "external_id": {
              "title": "An optional unique external identifier to use in conjunction with role to assume.",
              "description": "[external_id]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html",
              "type": [
                "string",
                "null"
              ]
            },
            "imds": {
              "description": "IMDS based authentication configuration.",
              "default": {
                "max_attempts": 4,
                "connect_timeout_seconds": 1,
                "read_timeout_seconds": 1
              },
              "type": "object",
              "properties": {
                "connect_timeout_seconds": {
                  "description": "IMDS connection timeout, in seconds.",
                  "default": 1,
                  "$ref": "#/definitions/duration_seconds"
                },
                "max_attempts": {
                  "description": "Maximum number of IMDS retry attempts for fetching tokens and metadata.",
                  "default": 4,
                  "type": "integer",
                  "maximum": 4294967295.0,
                  "minimum": 0.0
                },
                "read_timeout_seconds": {
                  "description": "IMDS read timeout, in seconds.",
                  "default": 1,
                  "$ref": "#/definitions/duration_seconds"
                }
              }
            },
            "load_timeout_secs": {
              "title": "Timeout for assuming the role, in seconds.",
              "description": "Relevant when either the default credentials chain or `assume_role` is used.",
              "type": [
                "integer",
                "null"
              ],
              "maximum": 9007199254740991.0,
              "minimum": 0.0
            },
            "region": {
              "title": "AWS region to send STS requests to.",
              "description": "If not set, this defaults to the configured region\nfor the service itself.",
              "type": [
                "string",
                "null"
              ]
            }
          }
        },
        {
          "description": "Default authentication strategy, tries multiple strategies",
          "type": "object",
          "properties": {
            "imds": {
              "description": "IMDS based authentication",
              "default": {
                "max_attempts": 4,
                "connect_timeout_seconds": 1,
                "read_timeout_seconds": 1
              },
              "type": "object",
              "properties": {
                "connect_timeout_seconds": {
                  "description": "IMDS connection timeout, in seconds.",
                  "default": 1,
                  "$ref": "#/definitions/duration_seconds"
                },
                "max_attempts": {
                  "description": "Number of IMDS retries for fetching tokens and metadata.",
                  "default": 4,
                  "type": "integer",
                  "maximum": 4294967295.0,
                  "minimum": 0.0
                },
                "read_timeout_seconds": {
                  "description": "IMDS read timeout, in seconds.",
                  "default": 1,
                  "$ref": "#/definitions/duration_seconds"
                }
              }
            },
            "load_timeout_secs": {
              "title": "Timeout for successfully loading any credentials, in seconds.",
              "description": "Relevant when the default credentials chain or `assume_role` is used.",
              "type": [
                "integer",
                "null"
              ],
              "maximum": 9007199254740991.0,
              "minimum": 0.0
            },
            "region": {
              "title": "AWS region to send STS requests to.",
              "description": "If not set, this defaults to the configured region\nfor the service itself.",
              "type": [
                "string",
                "null"
              ]
            }
          }
        }
      ]
    },
    "http_auth": {
      "title": "HTTP authentication Strategy.",
      "description": "HTTP authentication should be used with HTTPS only, as the authentication credentials are passed as an\nHTTP header without any additional encryption beyond what is provided by the transport itself.",
      "oneOf": [
        {
          "type": "null"
        },
        {
          "title": "Configuration of the authentication strategy for HTTP requests.",
          "description": "HTTP authentication should be used with HTTPS only, as the authentication credentials are passed as an\nHTTP header without any additional encryption beyond what is provided by the transport itself.",
          "oneOf": [
            {
              "title": "Basic authentication.",
              "description": "The username and password are concatenated [base64][base64] encoded.\n\n[base64]: https://en.wikipedia.org/wiki/Base64",
              "type": "object",
              "required": [
                "password",
                "strategy",
                "user"
              ],
              "properties": {
                "password": {
                  "description": "The basic authentication password.",
                  "type": "string"
                }
              }
            },
            {
              "title": "Bearer authentication.",
              "description": "The bearer token value (OAuth2, JWT, etc.) is passed through as-is.",
              "type": "object",
              "required": [
                "strategy",
                "token"
              ],
              "properties": {
                "strategy": {
                  "title": "Bearer authentication.",
                  "description": "The bearer token value (OAuth2, JWT, etc.) is passed as-is.",
                  "const": "bearer"
                },
                "token": {
                  "description": "The bearer authentication token.",
                  "type": "string"
                }
              }
            }
          ]
        }
      ]
    },
    "kafka_auth": {
      "description": "Kafka Authentication Strategy.",
      "type": "object",
      "properties": {
        "enabled": {
          "title": "Enables SASL authentication.",
          "description": "Only `PLAIN`- and `SCRAM`-based mechanisms are supported when configuring SASL authentication using `sasl.*`. For\nother mechanisms, `librdkafka_options.*` must be used directly to configure other `librdkafka`-specific values.\nIf using `sasl.kerberos.*` as an example, where `*` is `service.name`, `principal`, `kinit.md`, etc., then\n`librdkafka_options.*` as a result becomes `librdkafka_options.sasl.kerberos.service.name`,\n`librdkafka_options.sasl.kerberos.principal`, etc.\n\nSee the [librdkafka documentation](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md) for details.\n\nSASL authentication is not supported on Windows.",
          "type": [
            "boolean",
            "null"
          ]
        },
        "mechanism": {
          "description": "The SASL mechanism to use.",
          "type": [
            "string",
            "null"
          ]
        },
        "password": {
          "description": "The SASL password.",
          "type": "string"
        },
        "username": {
          "description": "The SASL username.",
          "type": [
            "string",
            "null"
          ]
        }
      }
    },
    "nats_auth": {
      "description": "NATS Authentication Strategy.",
      "oneOf": [
        {
          "type": "null"
        },
        {
          "description": "Configuration of the authentication strategy when interacting with NATS.",
          "oneOf": [
            {
              "description": "Username/password authentication.",
              "type": "object",
              "required": [
                "strategy",
                "user_password"
              ],
              "properties": {
                "strategy": {
                  "description": "Username/password authentication.",
                  "const": "user_password"
                },
                "user_password": {
                  "description": "Username and password configuration.",
                  "type": "object",
                  "required": [
                    "password",
                    "user"
                  ],
                  "properties": {
                    "password": {
                      "description": "Password.",
                      "type": "string"
                    },
                    "user": {
                      "description": "Username.",
                      "type": "string"
                    }
                  }
                }
              }
            },
            {
              "description": "Token-based authentication.",
              "type": "object",
              "required": [
                "strategy",
                "token"
              ],
              "properties": {
                "strategy": {
                  "description": "Token-based authentication.",
                  "const": "token"
                },
                "token": {
                  "description": "The authentication token.",
                  "type": "object",
                  "required": [
                    "value"
                  ],
                  "properties": {
                    "value": {
                      "description": "The authentication token.",
                      "type": "string"
                    }
                  }
                }
              }
            },
            {
              "description": "File based credential authentication. (JWT-based)",
              "type": "object",
              "required": [
                "credentials_file",
                "strategy"
              ],
              "properties": {
                "credentials_file": {
                  "description": "Credentials file configuration.",
                  "type": "object",
                  "required": [
                    "path"
                  ],
                  "properties": {
                    "path": {
                      "description": "System path to credentials file.",
                      "type": "string"
                    }
                  }
                },
                "strategy": {
                  "description": "File based credential authentication. (JWT-based)",
                  "const": "credentials_file"
                }
              }
            },
            {
              "description": "Public/Private (NKey) authentication.",
              "type": "object",
              "required": [
                "nkey",
                "strategy"
              ],
              "properties": {
                "nkey": {
                  "description": "Public/Private NKeys configuration.",
                  "type": "object",
                  "required": [
                    "nkey",
                    "seed"
                  ],
                  "properties": {
                    "nkey": {
                      "title": "User.",
                      "description": "User or public key.",
                      "type": "string"
                    },
                    "seed": {
                      "title": "Seed.",
                      "description": "Seed or private key.",
                      "type": "string"
                    }
                  }
                },
                "strategy": {
                  "description": "NKey-based authentication.",
                  "const": "nkey"
                }
              }
            }
          ]
        }
      ]
    },
    "duration_seconds": {
      "description": "A span of time, in whole seconds.",
      "type": "integer",
      "maximum": 9007199254740991.0,
      "minimum": 0.0
    }
  }
}

Environment

- OS:
- Node:
- npm:

Anything else?

No response

@kevinehosford kevinehosford added bug needs triage Initial label given, to be assigned correct labels and assigned labels Feb 12, 2024
@nickgros
Copy link
Contributor

@kevinehosford Are you able to add the omitExtraData and liveOmit props? That may solve your problem. I think omitExtraData on its own may not be working here due to a bug

@nickgros nickgros added awaiting response and removed needs triage Initial label given, to be assigned correct labels and assigned labels Feb 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants