Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New component: exceptionsconnector #17272

Closed
2 tasks
marctc opened this issue Dec 27, 2022 · 10 comments
Closed
2 tasks

New component: exceptionsconnector #17272

marctc opened this issue Dec 27, 2022 · 10 comments
Labels
Accepted Component New component has been sponsored Stale

Comments

@marctc
Copy link
Contributor

marctc commented Dec 27, 2022

The purpose and use-cases of the new component

The purpose of this connector component is to extract metrics and logs from recorded exceptions from spans. It will generate a counter (exceptions_total) where exceptions are aggregated by default by exception type and exception message. Aggregation could be configurable by the user.

metric output

exceptions_total{exception_message="list index out of range", exception_type="IndexError", service_name="test", span_kind="SPAN_KIND_SERVER", status_code="STATUS_CODE_ERROR"}  4

exceptions_total{exception_message="local variable 'a' referenced before assignment", exception_type="UnboundLocalError", service_name="test", span_kind="SPAN_KIND_SERVER", status_code="STATUS_CODE_ERROR"}  6

log output

{
  "severity": "ERROR",
  "attributes": {
    "exception.message": "division by zero",
    "exception.stacktrace": "Traceback (most recent call last):\n  File \"/usr/local/lib/python3.10/site-packages/opentelemetry/trace/__init__.py\", line 541, in use_span\n    yield span\n  File \"/usr/local/lib/python3.10/site-packages/django/core/handlers/exception.py\", line 55, in inner\n    response = get_response(request)\n  File \"/usr/local/lib/python3.10/site-packages/django/core/handlers/base.py\", line 197, in _get_response\n    response = wrapped_callback(request, *callback_args, **callback_kwargs)\n  File \"/code/./example/views.py\", line 24, in index\n    random.choice([f0, f1, f2, f3])()\n  File \"/code/./example/views.py\", line 7, in f0\n    a = 1/0\nZeroDivisionError: division by zero\n",
    "exception.type": "ZeroDivisionError",
    "http.flavor": "1.1",
    "http.host": "127.0.0.1:8000",
    "http.method": "GET",
    "http.scheme": "http",
    "http.server_name": "644f31ce42a5",
    "http.status_code": "",
    "http.target": "/",
    "http.user_agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:107.0) Gecko/20100101 Firefox/107.0",
    "service.name": "faulty"
  }
}

Example configuration for the component

receivers:
  otlp:
    protocols:
      grpc:

exporters:
  otlp:
    endpoint: htp://localhost:4317
  prometheusremotewrite:
    endpoint: http://prometheus:9090/api/v1/write
  loki:
    endpoint: http://loki:3100/loki/api/v1/push

connectors:
  exceptions: 

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [otlp, exceptions]
    metrics:
      receivers: [exceptions]
      exporters: [prometheusremotewrite]
    logs:
      receivers: [exceptions]
      exporters: [loki]

Telemetry data types supported

traces

Is this a vendor-specific component?

  • This is a vendor-specific component
  • If this is a vendor-specific component, I am proposing to contribute this as a representative of the vendor.

Sponsor (optional)

@jpkrohling

Additional context

No response

@marctc marctc added the needs triage New item requiring triage label Dec 27, 2022
@djaglowski
Copy link
Member

This is a good use case for a "connector", a new type of component that acts as both an exporter and receiver. You can read about the proposal here. The implementation of the framework is coming along, hopefully to be completed in the next month or so.

@jpkrohling jpkrohling added the Accepted Component New component has been sponsored label Jan 11, 2023
@jpkrohling
Copy link
Member

I'm willing to sponsor this as a connector.

@codeboten codeboten removed the needs triage New item requiring triage label Jan 11, 2023
@kovrus
Copy link
Member

kovrus commented Jan 11, 2023

Maybe worth merging it with #17277

@djaglowski
Copy link
Member

Maybe worth merging it with #17277

Yes, good point. This is very much possible with connectors.

@github-actions
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

@github-actions github-actions bot added the Stale label Mar 13, 2023
@djaglowski
Copy link
Member

Following up on this, a couple notes:

  1. Connectors can be implemented and used at this point, so there are no blockers there.
  2. As noted above, it should be possible to merge this proposal with New component: exceptionlogsprocessor #17277, to have an exceptionsconnector that consumes traces and emits metrics and/or logs.

Let me know if you have any questions about implementing this.

@marctc
Copy link
Contributor Author

marctc commented Mar 13, 2023

@djaglowski thanks for the input. That's exactly what I'm planing to do. Is connectors API ready to be used?

@djaglowski
Copy link
Member

The connectors API is ready. The count connector is a reasonable point of reference for how to use it with multiple signal types.

@marctc marctc changed the title New component: exceptionmetricsprocessor New component: exceptionsconnector Apr 24, 2023
@github-actions github-actions bot removed the Stale label May 26, 2023
jpkrohling added a commit that referenced this issue Jul 19, 2023
**Description:** Added connector to create metrics from recorded
exceptions. The implementation
is heavily inspired from
[spanmetricsconnector](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/connector/spanmetricsconnector).

**Link to tracking Issue:** #17272  

**Documentation:** Added examples in README.

---------

Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de>
@github-actions
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

@github-actions github-actions bot added the Stale label Jul 26, 2023
@marctc
Copy link
Contributor Author

marctc commented Jul 26, 2023

Done in #17277

@marctc marctc closed this as completed Jul 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Accepted Component New component has been sponsored Stale
Projects
None yet
Development

No branches or pull requests

5 participants