Structlog #5687

snaselj · 2024-05-08T15:35:28Z

Closes NaN

What's Changed

Added structlog logging.
Added NAUTOBOT_LOG_DEBUG_DB environment variable to enable Django database logs.
Added NAUTOBOT_LOG_PLAIN environment variable to switch to plaintext structlog output from JSON.

To Do

Open a new pull request to cookiecutter-nautobot-app repository to update development/nautobot_config.py file accordingly
This update, to be working properly in apps development environment, requires change to apps development/nautobot_config.py:

e.g. https://github.com/nautobot/nautobot-app-ssot/blob/develop/development/nautobot_config.py#L91

Here we should assign LOGGING only when it's not already defined to keep backward compatibility.

glennmatthews · 2024-05-08T20:11:05Z

Can you provide some examples of the resulting Nautobot logs both with and without DEBUG enabled?

glennmatthews

Since this is a non-trivial feature, would you please target the PR to next (2.3.0) rather than develop (2.2.x)?

glennmatthews · 2024-05-08T20:14:14Z

nautobot/core/settings.py

+    INSTALLED_APPS,
+    MIDDLEWARE,


Do these work as intended when adding additional apps via EXTRA_INSTALLED_APPS (https://docs.nautobot.com/projects/core/en/stable/user-guide/administration/configuration/optional-settings/#extra-applications) and/or when a Nautobot App adds its own middleware? (https://docs.nautobot.com/projects/core/en/stable/development/apps/api/nautobot-app-config/?h=middleware#optional-nautobotappconfig-attributes)

Works well with both EXTRA_INSTALLED_APPS and EXTRA_MIDDLEWARE.

glennmatthews · 2024-05-08T20:15:20Z

nautobot/core/settings.py

+    debug_db=is_truthy(os.getenv("NAUTOBOT_LOG_DEBUG_DB", "False")),
+    plain_format=is_truthy(os.getenv("NAUTOBOT_LOG_PLAIN", "False")),


If these are only configurable via environment variables (not ideal!) then they should be documented in nautobot/docs/user-guide/administration/configuration/optional-settings.md under the "Environment-Variable-Only Settings" header.

What is a better way to configure these? Note, we need to process these configuration settings into LOGGING Django settings attribute.

Ideally they'd be settable directly via Python in any nautobot_config.py as well without needing to manipulate os.environ.

changes/5687.housekeeping

nautobot/core/settings_funcs.py

glennmatthews · 2024-05-08T20:17:43Z

nautobot/core/settings_funcs.py

+    if "test" in sys.argv:
+        django_logging["handlers"] = {
+            "null_handler": {
+                "level": "INFO",


Not log_level?

level is the correct key here.

sorry, I meant instead of "INFO" being hard-coded. Same comment below.

For testing, it was hardcoded to INFO in the original settings, so this keeps the same behavior.

As it is using "class": "logging.NullHandler", it disables log anyway.

glennmatthews · 2024-05-08T20:17:51Z

nautobot/core/settings_funcs.py

+        }
+        for logger in django_logging["loggers"].values():
+            logger["handlers"] = ["null_handler"]
+            logger["level"] = "INFO"


Not log_level?

level is the correct key here.

nautobot/core/settings_funcs.py

glennmatthews · 2024-05-08T20:20:07Z

nautobot/core/settings_funcs.py

+    if debug_db:
+        django_logging["loggers"]["django.db.backends"] = {"level": "DEBUG"}


Why is this specific logger handled specially?

This is driven by NAUTOBOT_LOG_DEBUG_DB environment variable, inspired by management services configuration.

Added info about new environment variables to changes/5687.added.

And the answer to why is to be able to selectively enable database queries logging.

snaselj · 2024-05-09T08:08:17Z

Since this is a non-trivial feature, would you please target the PR to next (2.3.0) rather than develop (2.2.x)?

Changed base branch to next.

development/nautobot_config.py

snaselj · 2024-05-09T12:56:23Z

Can you provide some examples of the resulting Nautobot logs both with and without DEBUG enabled?

Log examples with:

NAUTOBOT_DEBUG=1
NAUTOBOT_LOG_PLAIN=1

nautobot-1  | 2024-05-09T12:40:58.089576Z [debug    ] Loading example_app!           [nautobot.extras.plugins.utils]
nautobot-1  | 2024-05-09T12:40:58.090570Z [info     ] Nautobot initialized!          [nautobot]
nautobot-1  | 2024-05-09T12:41:19.070546Z [info     ] request_started                [django_structlog.middlewares.request] ip=10.166.35.2 request=GET /login/?next=/ request_id=1e0de46b-570e-4bc5-bcff-c7c9731bc53f user_agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 user_id=None
nautobot-1  | 2024-05-09T12:41:19.165370Z [info     ] request_finished               [django_structlog.middlewares.request] code=200 ip=10.166.35.2 request=GET /login/?next=/ request_id=1e0de46b-570e-4bc5-bcff-c7c9731bc53f user_id=None
nautobot-1  | 2024-05-09T12:41:19.347753Z [info     ] request_started                [django_structlog.middlewares.request] ip=10.166.35.2 request=GET /template.css request_id=8832ce6f-ab1d-47b9-9169-dc2aae519438 user_agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 user_id=None
nautobot-1  | 2024-05-09T12:41:19.348687Z [info     ] request_finished               [django_structlog.middlewares.request] code=200 ip=10.166.35.2 request=GET /template.css request_id=8832ce6f-ab1d-47b9-9169-dc2aae519438 user_id=None
nautobot-1  | 2024-05-09T12:43:50.138837Z [info     ] request_started                [django_structlog.middlewares.request] ip=10.166.35.2 request=POST /login/ request_id=114b2ec7-5bc9-4d03-bca3-aef80d66371c user_agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 user_id=None
nautobot-1  | 2024-05-09T12:43:50.212636Z [debug    ] Login form validation was successful [nautobot.auth.login] ip=10.166.35.2 request_id=114b2ec7-5bc9-4d03-bca3-aef80d66371c user_id=None
nautobot-1  | 2024-05-09T12:43:50.229456Z [info     ] User admin successfully authenticated [nautobot.auth.login] ip=10.166.35.2 request_id=114b2ec7-5bc9-4d03-bca3-aef80d66371c user_id=None
nautobot-1  | 2024-05-09T12:43:50.236647Z [debug    ] Redirecting user to /          [nautobot.auth.login] ip=10.166.35.2 request_id=114b2ec7-5bc9-4d03-bca3-aef80d66371c user_id=None
nautobot-1  | 2024-05-09T12:43:50.236735Z [info     ] request_finished               [django_structlog.middlewares.request] code=302 ip=10.166.35.2 request=POST /login/ request_id=114b2ec7-5bc9-4d03-bca3-aef80d66371c user_id=99963881-7ce9-492b-8ec7-4c330ee1b799
nautobot-1  | 2024-05-09T12:43:50.350696Z [info     ] request_started                [django_structlog.middlewares.request] ip=10.166.35.2 request=GET / request_id=1fbd0806-d66c-41aa-b653-73a2d55a1e65 user_agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 user_id=99963881-7ce9-492b-8ec7-4c330ee1b799
nautobot-1  | 2024-05-09T12:43:50.369314Z [debug    ] Skipping release check; RELEASE_CHECK_URL not defined [nautobot.core.releases] ip=10.166.35.2 request_id=1fbd0806-d66c-41aa-b653-73a2d55a1e65 user_id=99963881-7ce9-492b-8ec7-4c330ee1b799
nautobot-1  | 2024-05-09T12:43:50.564229Z [info     ] request_finished               [django_structlog.middlewares.request] code=200 ip=10.166.35.2 request=GET / request_id=1fbd0806-d66c-41aa-b653-73a2d55a1e65 user_id=99963881-7ce9-492b-8ec7-4c330ee1b799
nautobot-1  | 2024-05-09T12:43:51.020018Z [info     ] request_started                [django_structlog.middlewares.request] ip=10.166.35.2 request=GET /template.css request_id=3415adf5-cecd-4a88-b5f6-9a27aecce293 user_agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 user_id=99963881-7ce9-492b-8ec7-4c330ee1b799
nautobot-1  | 2024-05-09T12:43:51.021238Z [info     ] request_finished               [django_structlog.middlewares.request] code=200 ip=10.166.35.2 request=GET /template.css request_id=3415adf5-cecd-4a88-b5f6-9a27aecce293 user_id=99963881-7ce9-492b-8ec7-4c330ee1b799

With:

NAUTOBOT_DEBUG=0
NAUTOBOT_LOG_PLAIN=1

nautobot-1  | 2024-05-09T12:49:40.638786Z [info     ] Nautobot initialized!          [nautobot]
nautobot-1  | 2024-05-09T12:49:43.592881Z [info     ] request_started                [django_structlog.middlewares.request] ip=10.166.35.2 request=GET /login/?next=/ request_id=dcb6c36d-cc81-4ffe-a599-ae4da453c151 user_agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 user_id=None
nautobot-1  | 2024-05-09T12:49:43.607732Z [info     ] request_finished               [django_structlog.middlewares.request] code=200 ip=10.166.35.2 request=GET /login/?next=/ request_id=dcb6c36d-cc81-4ffe-a599-ae4da453c151 user_id=None
nautobot-1  | 2024-05-09T12:49:43.723497Z [info     ] request_started                [django_structlog.middlewares.request] ip=10.166.35.2 request=GET /template.css request_id=ba119d6d-1248-483f-86ff-38211cc029d6 user_agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 user_id=None
nautobot-1  | 2024-05-09T12:49:43.724573Z [info     ] request_finished               [django_structlog.middlewares.request] code=200 ip=10.166.35.2 request=GET /template.css request_id=ba119d6d-1248-483f-86ff-38211cc029d6 user_id=None
nautobot-1  | 2024-05-09T12:49:46.811718Z [info     ] request_started                [django_structlog.middlewares.request] ip=10.166.35.2 request=POST /login/ request_id=f5506374-2cd6-4e8f-b2a9-84c037e44e53 user_agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 user_id=None
nautobot-1  | 2024-05-09T12:49:46.897887Z [info     ] User admin successfully authenticated [nautobot.auth.login] ip=10.166.35.2 request_id=f5506374-2cd6-4e8f-b2a9-84c037e44e53 user_id=None
nautobot-1  | 2024-05-09T12:49:46.905343Z [info     ] request_finished               [django_structlog.middlewares.request] code=302 ip=10.166.35.2 request=POST /login/ request_id=f5506374-2cd6-4e8f-b2a9-84c037e44e53 user_id=99963881-7ce9-492b-8ec7-4c330ee1b799
nautobot-1  | 2024-05-09T12:49:47.025006Z [info     ] request_started                [django_structlog.middlewares.request] ip=10.166.35.2 request=GET / request_id=4f66f8d9-ec7e-4dfc-a378-759b205789bb user_agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 user_id=99963881-7ce9-492b-8ec7-4c330ee1b799
nautobot-1  | 2024-05-09T12:49:47.079070Z [info     ] request_finished               [django_structlog.middlewares.request] code=200 ip=10.166.35.2 request=GET / request_id=4f66f8d9-ec7e-4dfc-a378-759b205789bb user_id=99963881-7ce9-492b-8ec7-4c330ee1b799
nautobot-1  | 2024-05-09T12:49:47.195770Z [info     ] request_started                [django_structlog.middlewares.request] ip=10.166.35.2 request=GET /template.css request_id=5205c18e-a0f6-41a6-be2a-9cd663e50aba user_agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 user_id=99963881-7ce9-492b-8ec7-4c330ee1b799
nautobot-1  | 2024-05-09T12:49:47.196435Z [info     ] request_finished               [django_structlog.middlewares.request] code=200 ip=10.166.35.2 request=GET /template.css request_id=5205c18e-a0f6-41a6-be2a-9cd663e50aba user_id=99963881-7ce9-492b-8ec7-4c330ee1b799

With:

NAUTOBOT_DEBUG=0
NAUTOBOT_LOG_PLAIN=0

nautobot-1  | {"event": "Nautobot initialized!", "level": "info", "logger": "nautobot", "timestamp": "2024-05-09T12:51:43.677022Z"}
nautobot-1  | {"request": "GET /login/?next=/", "user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36", "event": "request_started", "level": "info", "logger": "django_structlog.middlewares.request", "user_id": null, "ip": "10.166.35.2", "request_id": "9b9f00e7-7cbc-431f-a9cf-a484b9d1bc12", "timestamp": "2024-05-09T12:51:49.244073Z"}
nautobot-1  | {"code": 200, "request": "GET /login/?next=/", "event": "request_finished", "level": "info", "logger": "django_structlog.middlewares.request", "user_id": null, "ip": "10.166.35.2", "request_id": "9b9f00e7-7cbc-431f-a9cf-a484b9d1bc12", "timestamp": "2024-05-09T12:51:49.258771Z"}
nautobot-1  | {"request": "GET /template.css", "user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36", "event": "request_started", "level": "info", "logger": "django_structlog.middlewares.request", "user_id": null, "ip": "10.166.35.2", "request_id": "85946f20-f9a8-4fa6-a60b-0e60465cac04", "timestamp": "2024-05-09T12:51:49.402089Z"}
nautobot-1  | {"code": 200, "request": "GET /template.css", "event": "request_finished", "level": "info", "logger": "django_structlog.middlewares.request", "user_id": null, "ip": "10.166.35.2", "request_id": "85946f20-f9a8-4fa6-a60b-0e60465cac04", "timestamp": "2024-05-09T12:51:49.402872Z"}
nautobot-1  | {"request": "POST /login/", "user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36", "event": "request_started", "level": "info", "logger": "django_structlog.middlewares.request", "user_id": null, "ip": "10.166.35.2", "request_id": "ed57d881-cad4-4240-8492-1858cb6e5aca", "timestamp": "2024-05-09T12:51:53.628068Z"}
nautobot-1  | {"event": "User admin successfully authenticated", "level": "info", "logger": "nautobot.auth.login", "user_id": null, "ip": "10.166.35.2", "request_id": "ed57d881-cad4-4240-8492-1858cb6e5aca", "timestamp": "2024-05-09T12:51:53.714045Z"}
nautobot-1  | {"code": 302, "request": "POST /login/", "event": "request_finished", "level": "info", "logger": "django_structlog.middlewares.request", "user_id": "99963881-7ce9-492b-8ec7-4c330ee1b799", "ip": "10.166.35.2", "request_id": "ed57d881-cad4-4240-8492-1858cb6e5aca", "timestamp": "2024-05-09T12:51:53.721586Z"}
nautobot-1  | {"request": "GET /", "user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36", "event": "request_started", "level": "info", "logger": "django_structlog.middlewares.request", "user_id": "99963881-7ce9-492b-8ec7-4c330ee1b799", "ip": "10.166.35.2", "request_id": "50b8e78f-92f9-41d0-8c54-69948fa702e6", "timestamp": "2024-05-09T12:51:53.902784Z"}
nautobot-1  | {"code": 200, "request": "GET /", "event": "request_finished", "level": "info", "logger": "django_structlog.middlewares.request", "user_id": "99963881-7ce9-492b-8ec7-4c330ee1b799", "ip": "10.166.35.2", "request_id": "50b8e78f-92f9-41d0-8c54-69948fa702e6", "timestamp": "2024-05-09T12:51:53.953947Z"}
nautobot-1  | {"request": "GET /template.css", "user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36", "event": "request_started", "level": "info", "logger": "django_structlog.middlewares.request", "user_id": "99963881-7ce9-492b-8ec7-4c330ee1b799", "ip": "10.166.35.2", "request_id": "633c1ec3-b2ac-4bb2-bd95-447e2b6419c9", "timestamp": "2024-05-09T12:51:54.073128Z"}
nautobot-1  | {"code": 200, "request": "GET /template.css", "event": "request_finished", "level": "info", "logger": "django_structlog.middlewares.request", "user_id": "99963881-7ce9-492b-8ec7-4c330ee1b799", "ip": "10.166.35.2", "request_id": "633c1ec3-b2ac-4bb2-bd95-447e2b6419c9", "timestamp": "2024-05-09T12:51:54.073773Z"}

jvanderaa · 2024-05-10T22:12:42Z

Is this a breaking change? Will the defaults have logs in the same output as before? I'm thinking about security organizations that may have parsers of logs set up...

lampwins · 2024-05-10T23:46:53Z

I don’t feel comfortable with the size and scope of this for 2.3. When we spoke about this, my understanding was we were going to add structlog as a core dependency only. Any configuration can go in nautobot_config.py.

snaselj · 2024-05-13T07:50:31Z

Is this a breaking change? Will the defaults have logs in the same output as before? I'm thinking about security organizations that may have parsers of logs set up...

By default, log output is different now, using structlog JSON formatting.

For instances rewriting LOGGING configuration key in a custom nautobot_config.py, nothing should change.

snaselj · 2024-05-13T07:56:56Z

I don’t feel comfortable with the size and scope of this for 2.3. When we spoke about this, my understanding was we were going to add structlog as a core dependency only. Any configuration can go in nautobot_config.py.

@lampwins

structlog is a core dependency, not sure what should be changed here.

Separated structlog configuration to core/settings_funcs.py, to be reusable. Do you want to move everything to core/settings.py instead?

Added NAUTOBOT_LOG_PLAIN to allow switching to plaintext structlog logging. Do you want to remove this settings?

Added NAUTOBOT_LOG_DEBUG_DB to allow enabling SQL logs. Do you want to remove this settings?

glennmatthews · 2024-05-13T17:58:41Z

As I understand it the approach we would prefer is to add structlog as a dependency, but not to enable it in 2.3 unless this can be done in such a way as to not introduce any behavior change in the logging output for any existing deployments that are using Nautobot's provided default logging configuration.

snaselj · 2024-05-20T07:11:16Z

@glennmatthews

Updated based on the latest comments, ready for review now.

nautobot/core/settings_funcs.py

nautobot/docs/development/core/settings.md

changes/5687.dependencies

development/nautobot_config.py

nautobot/core/settings_funcs.py

glennmatthews · 2024-05-21T13:16:46Z

nautobot/docs/development/core/settings.md

@@ -85,3 +85,30 @@ Markdown rendering is supported for the `description`, `details`, and `default_l
 ### Technical Details of Settings Documentation

 The `optional-settings.md` and `required-settings.md` files are rendered as Jinja2 templates via [`Mkdocs-Macros`](https://mkdocs-macros-plugin.readthedocs.io/en/latest/). The file `nautobot/docs/macros.py` is responsible for loading `settings.yaml` into the template context for rendering. `mkdocs.yml` instructs Mkdocs-Macros to run that file at documentation rendering time. The macros and templating are *not* enabled for all documentation by default - instead, only the files with `render_macros: true` in their headers will be templated.
+
+## Structlog Configuration


This settings.md file is developer-facing not end-user-facing. Can you add this content to nautobot/core/settings.yaml under the LOGGING details section instead?

Moved as proposed.

snaselj self-assigned this May 8, 2024

glennmatthews reviewed May 8, 2024

View reviewed changes

snaselj changed the base branch from develop to next May 9, 2024 08:06

feat: Structlog

f67f724

snaselj force-pushed the u/snaselj-structlog branch from 0c13b1e to f67f724 Compare May 9, 2024 08:33

snaselj added 2 commits May 9, 2024 09:51

fix: Use exc_info renderer properly.

850b790

fix: Tests

02cd30f

snaselj commented May 9, 2024

View reviewed changes

development/nautobot_config.py Outdated Show resolved Hide resolved

snaselj added 2 commits May 9, 2024 11:12

fix: Tests

174c48c

fix: Tests

1d3d500

snaselj marked this pull request as ready for review May 9, 2024 12:56

snaselj requested a review from glennmatthews May 9, 2024 12:56

Merge remote-tracking branch 'origin/next' into u/snaselj-structlog

c3c5b72

snaselj added 3 commits May 20, 2024 06:39

Merge remote-tracking branch 'origin/next' into u/snaselj-structlog

adbfdd4

chore: PR feedback

17792a7

fixes

782aea2

chore: Revert tests

f594d72

glennmatthews reviewed May 20, 2024

View reviewed changes

snaselj added 2 commits May 21, 2024 07:15

Merge remote-tracking branch 'origin/next' into u/snaselj-structlog

b351cc2

fix: PR feedback

704b2a1

glennmatthews reviewed May 21, 2024

View reviewed changes

snaselj added 2 commits May 22, 2024 06:16

Merge remote-tracking branch 'origin/next' into u/snaselj-structlog

4cf8fd2

chore: Move structlog documentation

2020dd9

glennmatthews approved these changes May 22, 2024

View reviewed changes

HanlinMiao approved these changes May 23, 2024

View reviewed changes

glennmatthews merged commit ded1ce7 into next May 23, 2024
17 checks passed

glennmatthews deleted the u/snaselj-structlog branch May 23, 2024 17:23

glennmatthews mentioned this pull request May 23, 2024

Implement structlog Support #4087

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Structlog #5687

Structlog #5687

snaselj commented May 8, 2024 •

edited

glennmatthews commented May 8, 2024

glennmatthews left a comment

glennmatthews May 8, 2024

snaselj May 9, 2024

glennmatthews May 8, 2024

snaselj May 9, 2024

glennmatthews May 13, 2024

glennmatthews May 8, 2024

snaselj May 9, 2024

glennmatthews May 9, 2024

snaselj May 10, 2024

glennmatthews May 8, 2024

snaselj May 9, 2024

glennmatthews May 8, 2024

snaselj May 9, 2024

snaselj May 9, 2024

snaselj commented May 9, 2024

snaselj commented May 9, 2024

jvanderaa commented May 10, 2024

lampwins commented May 10, 2024

snaselj commented May 13, 2024

snaselj commented May 13, 2024

glennmatthews commented May 13, 2024

snaselj commented May 20, 2024

glennmatthews May 21, 2024

snaselj May 22, 2024

		debug_db=is_truthy(os.getenv("NAUTOBOT_LOG_DEBUG_DB", "False")),
		plain_format=is_truthy(os.getenv("NAUTOBOT_LOG_PLAIN", "False")),

		if debug_db:
		django_logging["loggers"]["django.db.backends"] = {"level": "DEBUG"}

Structlog #5687

Structlog #5687

Conversation

snaselj commented May 8, 2024 • edited

Closes NaN

What's Changed

To Do

glennmatthews commented May 8, 2024

glennmatthews left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

snaselj commented May 9, 2024

snaselj commented May 9, 2024

jvanderaa commented May 10, 2024

lampwins commented May 10, 2024

snaselj commented May 13, 2024

snaselj commented May 13, 2024

glennmatthews commented May 13, 2024

snaselj commented May 20, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

snaselj commented May 8, 2024 •

edited