Skip to content

Conversation

@Josephasafg
Copy link
Contributor

Added support for the following methods -

  • set_verbose - This will more log sensitive data to your log like env variable and secrets
  • set_debug - Another way to change log level to DEBUG

@Josephasafg Josephasafg requested a review from miri-bar June 24, 2024 08:31
@github-actions github-actions bot added the feature New functionality label Jun 24, 2024
@miri-bar
Copy link
Contributor

Maybe it's worth checking if we can use a custom logging handler to do this logic? Maybe in the emit function of the handler?

httpx_logger = logging.getLogger("httpx")


class CensorSecretsFormatter(logging.Formatter):
Copy link
Contributor

@pazshalev pazshalev Jun 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using logging.Filter instead of logging.Formatter, seems simpler IMO in this case, like so:

class SensitiveInfoFilter(logging.Filter):
    def filter(self, record):
        record.msg = re.sub(r'<PATTERN>', '*********', record.msg)
        return True


logger.addFilter(SensitiveInfoFilter())

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't quite what we need here, as loggingFilter filter the log so it won't be logged. I do want to log it, simply without some of the data that is being written

ai21/logger.py Outdated

def _censor_secrets(self, message: str) -> str:
# Regular expression to find the Authorization key and its value
pattern = r"('Authorization':\s*'[^']*'|'api-key':\s*'[^']*')"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're logging headers, we'd need to also add to the pattern X-Amz-Security-Token as this is one of the auth headers for aws

@github-actions github-actions bot added the lgtm Looks Good to Me label Jul 1, 2024
@Josephasafg Josephasafg merged commit 57b1ea9 into main Jul 1, 2024
@Josephasafg Josephasafg deleted the feat_log_verbosity branch July 1, 2024 10:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New functionality lgtm Looks Good to Me size:l

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants