feat(logs): add sentry log API + send first logs

[JoshuaMoelans]

First step towards #1210 (Merging into #1271)

Adds the initial logs API sentry_log_X(...) and constructs envelopes with the appropriate date & attributes. Currently sends these one at a time (for each API call), so next PR will be adding batching for logs.

The 'first logs' visible on Sentry

#skip-changelog

[github-actions]

	Messages
📖	Do not forget to update Sentry-docs with your feature once the pull request gets approved.

Generated by 🚫 dangerJS against 0dcc172

[JoshuaMoelans]

Look into why other SDKs are using this too (instead of parameter.name); could be useful to track the variable over time across log statements. Probably hard to parse this in Native though.

-> insofar no usage of sentry.message.parameter.X found where X is not an integer. Will have to ask around internally.

[JoshuaMoelans]

The Python SDK should be doing this for f-strings, but it's not showing up as parameter.X (see getsentry/sentry-python#4525).

As mentioned in #1272 (review), providing a separate interface that allows for these named parameters could be a potential improvement that will be part of the next Logs PR.

[JoshuaMoelans]

temporarily back to draft, seems like we have some memory leaks 🧠 💧

[JoshuaMoelans]

@sentry review

[seer-by-sentry]

On it! We are reviewing the PR and will provide feedback shortly.

[seer-by-sentry]

PR Description

This pull request introduces a new structured logging feature to the Sentry Native SDK. The goal is to enable more detailed and contextual logging within applications using the SDK, allowing for better debugging and diagnostics of issues reported to Sentry.

Click to see more

Key Technical Changes

The key changes include: 1) Addition of new sentry_log_* functions (trace, debug, info, warn, error, fatal) to the public API (include/sentry.h). 2) Implementation of the logging logic in src/sentry_logs.c, which constructs a Sentry log item from the log message and its arguments, enriching it with contextual information from the scope (trace ID, user information, OS context, etc.). 3) Creation of a new envelope item type 'log' to encapsulate the log data. 4) Modification of sentry_options to include a flag (enable_logs) to control whether logs are captured and sent. 5) Addition of unit tests (tests/unit/test_logs.c) to verify the basic functionality of the logging feature.

Architecture Decisions

The primary architectural decision is to treat logs as a separate item type within the Sentry envelope, similar to events and transactions. This allows for efficient transport and processing of log data alongside other Sentry data. The log messages are structured as Sentry values (objects and lists) to facilitate querying and analysis within the Sentry platform. The implementation uses variadic functions for the logging API, which requires careful handling of format strings to prevent security vulnerabilities. The decision was made to send logs immediately rather than batching them, with a TODO comment indicating that batching should be implemented later.

Dependencies and Interactions

This feature depends on the existing Sentry core functionality, including sentry_options, sentry_scope, sentry_envelope, and sentry_value. It interacts with the transport layer to send the log data to Sentry. The logging feature also relies on the OS context and user information from the scope. It adds a dependency on sentry_logs.h and sentry_logs.c.

Risk Considerations

Potential risks include: 1) Performance overhead of constructing and sending log messages, especially in high-volume scenarios. 2) Security vulnerabilities related to format string handling in the variadic logging functions. 3) Increased bandwidth usage due to the additional log data being sent to Sentry. 4) The immediate sending of logs without batching could lead to rate limiting issues. 5) The current implementation doesn't filter logs based on level, which could lead to excessive logging if trace level is enabled in production.

Notable Implementation Details

The construct_log function in src/sentry_logs.c is responsible for creating the structured log message. It parses the format string and extracts the arguments to create a dictionary of parameters. It also retrieves contextual information from the scope and options. The populate_message_parameters function handles the parsing of the format string and extraction of arguments. The code includes a TODO comment to implement batched log sending. The level_as_string function in src/sentry_logs.c and sentry_value.c and sentry__logger_describe function in src/sentry_logger.c need to be updated to include the SENTRY_LEVEL_TRACE case.

[supervacuus]

That is a great start! I might give this another go tomorrow, but there are already a couple of points to address.

I did not explicitly mention the alternative interface for the format parameters (except for a concrete point where it overlaps with this PR, at least in terms of design effect)

[supervacuus]

We must be cautious here: if you skip the length specifiers, a 64-bit value could be passed in. Even if the value is a 32-bit unsigned, there is no way the backend could know that this is an int32 containing a uint32 without an explicit protocol.

Essentially, if we accept unsigned integers of 32-bit, we'd have to store them in a double.

If we accept 64-bit unsigned integers, it must be clear that JSON will overflow numbers above 2^53-1

[supervacuus]

By the way, introducing sentry_value_t (a list of kv-objects) as a parameter to format strings would also introduce some normalization (or put the burden of providing parameters that map to the sentry payloads on the user, instead of the code having to guess) in terms of the data that users can pass in here.

[supervacuus]

Same as below... since you are skipping length, this could be a 64-bit signed integer.