Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Azure] Add input metrics to the azure-eventhub input #35739

Merged
merged 13 commits into from
Aug 15, 2023

Commits on Aug 14, 2023

  1. Draft input metrics about events and records

    In this first iteration, the metrics keep track of the following data
    types:
    
    - events
    - records
    
    Events are the event delivered by the event hub; each event usually
    contains a list of records.
    
    Records are the actual logs from Azure; the input creates one document
    in Elasticsearch for each record.
    
    This draft keeps track of the following conditions.
    
    Events:
    
    - received: the input received from an event from the Event Hub
    - sanitized: the event contains invalid JSON, and the input tried fixing it
    - deserialization failed: the event contains invalid JSON; sanitization was ineffective
    - processed: the input processed all the records in the event
    
    Records:
    
    - received: the input  unpacked a record from an event
    - serialization failed: failed to serialize the record for dispatching
    - processed: the input dispatched the record to the outlet
    zmoog committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    4c3d276 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    1e149b5 View commit details
    Browse the repository at this point in the history
  3. Fix linter objections

    zmoog committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    3508c13 View commit details
    Browse the repository at this point in the history
  4. Fix typos and stale comments

    zmoog committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    46792ef View commit details
    Browse the repository at this point in the history
  5. Set input ID with the config hash

    On v1 inputs, using the hash function of the config is a common
    practice.
    zmoog committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    a7d7dc3 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    6be69dd View commit details
    Browse the repository at this point in the history
  7. Add input metrics docs

    zmoog committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    b612d6d View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    7be9357 View commit details
    Browse the repository at this point in the history
  9. Move input ID and metrics closer

    This way it's more evident the two go along together. I am also adding
    comments for underline the ID will go away as we migrate the input
    to the input V2 API.
    zmoog committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    754337a View commit details
    Browse the repository at this point in the history
  10. Add a guard to unregister()

    zmoog committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    1649f79 View commit details
    Browse the repository at this point in the history
  11. Log major lifecycle events

    Adds a few more log message during major lifecycle events like
    failures in event publishing and input stopping.
    zmoog committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    9d48263 View commit details
    Browse the repository at this point in the history
  12. Update CHANGELOG

    zmoog committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    c934192 View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2023

  1. Configuration menu
    Copy the full SHA
    cbc4fde View commit details
    Browse the repository at this point in the history