Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(tracer): Support for external observability providers - Tracer #2342

Draft
wants to merge 46 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
06c9110
refactor to standalone tracer class
roger-zhangg May 29, 2023
5eab094
refactor tracer provider, add a xray provider
roger-zhangg May 29, 2023
cbd75f9
fix static checking error
roger-zhangg May 30, 2023
a6c6171
Merge branch 'develop' into tracer
roger-zhangg Jan 31, 2024
927710c
add_dd_poc
roger-zhangg Feb 6, 2024
e6b3a7c
first working POC for dd
roger-zhangg Feb 15, 2024
dde7b7f
fix close
roger-zhangg Feb 15, 2024
da8effc
add OPTL POC
roger-zhangg Feb 20, 2024
61f5757
add new base and xray provider
roger-zhangg Feb 29, 2024
0cb8463
add functional xray provider
roger-zhangg Mar 13, 2024
7f8bf6c
Merging from develop
leandrodamascena Mar 14, 2024
64dd749
Merging from develop
leandrodamascena Mar 14, 2024
f5fa320
Merging from develop
leandrodamascena Mar 15, 2024
52cab66
Merge branch 'develop' into tracer
leandrodamascena Mar 15, 2024
ef7c509
fix to pass test
roger-zhangg Mar 19, 2024
f66ad21
Merge branch 'tracer' of github.com:roger-zhangg/aws-lambda-powertool…
roger-zhangg Mar 19, 2024
2c3f42e
Merging from develop
leandrodamascena Mar 19, 2024
78051f6
fix mypy, move into folder
roger-zhangg Mar 20, 2024
bf16f6e
Merge branch 'develop' into tracer
roger-zhangg Mar 20, 2024
5bc0158
fix poetry
roger-zhangg Mar 20, 2024
fe9c763
add set_attribute in tracer
roger-zhangg Mar 21, 2024
d49e1cc
fix context datadog
roger-zhangg Mar 21, 2024
7fc17b3
Merge branch 'develop' into tracer
roger-zhangg Mar 21, 2024
5134c36
Merge remote-tracking branch 'upstream/develop' into tracer
leandrodamascena Mar 21, 2024
1b28653
Merging from develop
leandrodamascena Mar 21, 2024
b1a2b34
fix docstring, add conversion in otel
roger-zhangg Mar 25, 2024
ed1059b
fix typo
roger-zhangg Mar 25, 2024
4639bd4
add in_subsegment_async in dd
roger-zhangg Mar 25, 2024
012f8a8
remove discuss
roger-zhangg Mar 26, 2024
c9592c5
Merge branch 'develop' into tracer
roger-zhangg Mar 26, 2024
f0561cc
fix poetry
roger-zhangg Mar 26, 2024
33e0225
Merging from develop
leandrodamascena Apr 9, 2024
0ac3800
Merging from develop
leandrodamascena Apr 9, 2024
dede79c
merge from develop
roger-zhangg Apr 10, 2024
844d85b
address ruben comments
roger-zhangg Apr 10, 2024
cedb2af
change to new example
roger-zhangg Apr 10, 2024
ee869b3
change to new example
roger-zhangg Apr 10, 2024
68ee403
docstring in current base class
roger-zhangg Apr 11, 2024
07714e9
refactor set_attribute, fix test
roger-zhangg Apr 12, 2024
8712a30
Merge branch 'develop' of github.com:roger-zhangg/aws-lambda-powertoo…
roger-zhangg Apr 12, 2024
5fb7990
fix poetry
roger-zhangg Apr 12, 2024
aeaf6ff
remote print
roger-zhangg Apr 12, 2024
024db25
Addressing Ruben's feedback
leandrodamascena Apr 17, 2024
80739fd
Reveting possible breaking change
leandrodamascena Apr 17, 2024
d502dbf
Merging from develop
leandrodamascena Apr 17, 2024
7efe7e4
Merging from develop
leandrodamascena Apr 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 3 additions & 2 deletions aws_lambda_powertools/tracing/base.py
Expand Up @@ -5,17 +5,18 @@
from typing import Any, Generator, List, Optional, Sequence, Union


## MAINTENANCE: deprecated, kept for backwards compatibility until v3 is released
class BaseSegment(abc.ABC):
"""Holds common properties and methods on segment and subsegment."""

@abc.abstractmethod
def close(self, end_time: Optional[int] = None):
def close(self, end_time: Optional[float] = None):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a breaking change? Or are we assuming that no one is inheriting from BaseSegment today?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will deprecate this in Powertools v3. We're not using this class today, but we never know if customers are, so it's best to revert this change.

Reverted.

"""Close the trace entity by setting `end_time`
and flip the in progress flag to False.

Parameters
----------
end_time: int
end_time: float
Time in epoch seconds, by default current time will be used.
"""
roger-zhangg marked this conversation as resolved.
Show resolved Hide resolved

Expand Down
Empty file.
112 changes: 112 additions & 0 deletions aws_lambda_powertools/tracing/provider/base.py
@@ -0,0 +1,112 @@
import abc
from contextlib import asynccontextmanager, contextmanager
from typing import Any, AsyncGenerator, Generator, Sequence


class BaseSpan(abc.ABC):
"""A span represents a unit of work or operation within a trace.
Spans are the building blocks of Traces."""

@abc.abstractmethod
def set_attribute(self, key: str, value: Any, **kwargs) -> None:
"""Set an attribute for a span with a key-value pair.

Parameters
----------
key: str
Attribute key
value: Any
Attribute value
kwargs: Optional[dict]
Optional parameters
"""

@abc.abstractmethod
def record_exception(self, exception: BaseException, **kwargs):
"""Records an exception to this Span.

Parameters
----------
exception: Exception
Caught exception during the exectution of this Span
kwargs: Optional[dict]
Optional parameters
"""


class BaseProvider(abc.ABC):
roger-zhangg marked this conversation as resolved.
Show resolved Hide resolved
"""BaseProvider is an abstract base class that defines the expected behavior for tracing providers
used by Tracer. Inheriting classes must implement this interface to be compatible with Tracer.
"""

@abc.abstractmethod
@contextmanager
def trace(self, name: str, **kwargs) -> Generator[BaseSpan, None, None]:
"""Context manager for creating a new span and set it
as the current span in this tracer's context.

Exiting the context manager will call the span's end method,
as well as return the current span to its previous value by
returning to the previous context.

Parameters
----------
name: str
Span name
kwargs: Optional[dict]
Optional parameters to be propagated to the span
"""

@abc.abstractmethod
@asynccontextmanager
def trace_async(self, name: str, **kwargs) -> AsyncGenerator[BaseSpan, None]:
"""Async Context manager for creating a new span and set it
as the current span in this tracer's context.

Exiting the context manager will call the span's end method,
as well as return the current span to its previous value by
returning to the previous context.

Parameters
----------
name: str
Span name
kwargs: Optional[dict]
Optional parameters to be propagated to the span
"""

@abc.abstractmethod
def set_attribute(self, key: str, value: Any, **kwargs) -> None:
"""set attribute on current active span with a key-value pair.

Parameters
----------
key: str
attribute key
value: Any
attribute value
kwargs: Optional[dict]
Optional parameters to be propagated to the span
"""

@abc.abstractmethod
def patch(self, modules: Sequence[str]) -> None:
"""Instrument a set of given libraries if supported by provider
See specific provider for more detail

Exmaple
-------
tracer = Tracer(service="payment")
libraries = (['aioboto3',mysql])
# provider.patch will be called by tracer.patch
tracer.patch(libraries)

Parameters
----------
modules: Set[str]
Set of modules to be patched
"""

@abc.abstractmethod
def patch_all(self) -> None:
"""Instrument all supported libraries"""
141 changes: 141 additions & 0 deletions aws_lambda_powertools/tracing/provider/xray/xray_tracer.py
@@ -0,0 +1,141 @@
from __future__ import annotations

from contextlib import asynccontextmanager, contextmanager
from numbers import Number
from typing import Any, AsyncGenerator, Generator, Literal, Sequence, Union

from ....shared import constants
from ....shared.lazy_import import LazyLoader
from ..base import BaseProvider, BaseSpan
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're trying to use full imports now


aws_xray_sdk = LazyLoader(constants.XRAY_SDK_MODULE, globals(), constants.XRAY_SDK_MODULE)


class XraySpan(BaseSpan):
def __init__(self, subsegment):
self.subsegment = subsegment
self.add_subsegment = self.subsegment.add_subsegment
self.remove_subsegment = self.subsegment.remove_subsegment
self.put_annotation = self.subsegment.put_annotation
self.put_metadata = self.subsegment.put_metadata
self.add_exception = self.subsegment.add_exception
self.close = self.subsegment.close

def set_attribute(
self,
key: str,
value: Any,
category: Literal["Annotation", "Metadata", "Auto"] = "Auto",
**kwargs,
) -> None:
"""
Set attribute on this span with a key-value pair.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Set attribute on this span with a key-value pair.
Set an attribute on this span with a key-value pair.


Parameters
----------
key : str
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
key : str
key: str

attribute key
value : Any
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
value : Any
value: Any

Value for attribute
category : Literal["Annotation","Metadata","Auto"] = "Auto"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
category : Literal["Annotation","Metadata","Auto"] = "Auto"
category: Literal["Annotation","Metadata","Auto"] = "Auto"

This parameter specifies the category of attribute to set.
- **"Annotation"**: Sets the attribute as an Annotation.
- **"Metadata"**: Sets the attribute as Metadata.
- **"Auto" (default)**: Automatically determines the attribute
type based on its value.

kwargs: Optional[dict]
Optional parameters to be passed to provider.set_attributes
"""
if category == "Annotation":
self.put_annotation(key=key, value=value)
return

if category == "Metadata":
self.put_metadata(key=key, value=value, namespace=kwargs.get("namespace", "dafault"))
return

# Auto
if isinstance(value, (str, Number, bool)):
self.put_annotation(key=key, value=value)
return

# Auto & not in (str, Number, bool)
self.put_metadata(key=key, value=value, namespace=kwargs.get("namespace", "dafault"))

def record_exception(self, exception: BaseException, **kwargs):
stack = aws_xray_sdk.core.utils.stacktrace.get_stacktrace()
self.add_exception(exception=exception, stack=stack)


class XrayProvider(BaseProvider):
def __init__(self, xray_recorder=None):
if not xray_recorder:
from aws_xray_sdk.core import xray_recorder
self.recorder = xray_recorder
self.in_subsegment = self.recorder.in_subsegment
self.in_subsegment_async = self.recorder.in_subsegment_async

@contextmanager
def trace(self, name: str, **kwargs) -> Generator[XraySpan, None, None]:
with self.in_subsegment(name=name, **kwargs) as sub_segment:
yield XraySpan(subsegment=sub_segment)

@asynccontextmanager
async def trace_async(self, name: str, **kwargs) -> AsyncGenerator[XraySpan, None]:
async with self.in_subsegment_async(name=name, **kwargs) as subsegment:
yield XraySpan(subsegment=subsegment)

def set_attribute(
self,
key: str,
value: Any,
category: Literal["Annotation", "Metadata", "Auto"] = "Auto",
**kwargs,
) -> None:
"""
Set attribute on the current active span with a key-value pair.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Set attribute on the current active span with a key-value pair.
Set an attribute on the current active span with a key-value pair.


Parameters
----------
key : str
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
key : str
key: str

attribute key
value : Any
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
value : Any
value: Any

Value for attribute
category : Literal["Annotation","Metadata","Auto"] = "Auto"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
category : Literal["Annotation","Metadata","Auto"] = "Auto"
category: Literal["Annotation","Metadata","Auto"] = "Auto"

This parameter specifies the type of attribute to set.
- **"Annotation"**: Sets the attribute as an Annotation.
- **"Metadata"**: Sets the attribute as Metadata.
- **"Auto" (default)**: Automatically determines the attribute
type based on its value.

kwargs: Optional[dict]
Optional parameters to be passed to provider.set_attributes
"""
if category == "Annotation":
self.put_annotation(key=key, value=value)
return

if category == "Metadata":
self.put_metadata(key=key, value=value, namespace=kwargs.get("namespace", "dafault"))
return

# Auto
if isinstance(value, (str, Number, bool)):
self.put_annotation(key=key, value=value)
return

# Auto & not in (str, Number, bool)
self.put_metadata(key=key, value=value, namespace=kwargs.get("namespace", "dafault"))

def put_annotation(self, key: str, value: Union[str, Number, bool]) -> None:
return self.recorder.put_annotation(key=key, value=value)

def put_metadata(self, key: str, value: Any, namespace: str = "default") -> None:
return self.recorder.put_metadata(key=key, value=value, namespace=namespace)

def patch(self, modules: Sequence[str]) -> None:
return aws_xray_sdk.core.patch(modules)

def patch_all(self) -> None:
return aws_xray_sdk.core.patch_all()