fix: handle text types in _set_str_tag #2218

majorgreys · 2021-03-23T21:59:11Z

The addition of _set_str_tag in #1804 exposed cases in Python 2 where tag values were being set that were not of type unicode which was resulting in raising a UnicodeDecodingError exception. We handle this better by making use of six.ensure_text in _set_str_tag as well as still catching the exception raised from it if the value is of a type that is not coercible. But this requires we ensure that the function is not called with types other than those that can be coerced with ensure_text, which means changing some of its uses in our integrations where we were assuming that we could pass stringify-able arguments.

Kyle-Verhoog · 2021-03-23T22:10:30Z

tests/tracer/test_span.py

+def test_span_bad_unicode_set_tag():
+    span = Span(None, None)
+    span.set_tag("key", "🤔")
+    span._set_str_tag("key", "🤔")


I think we should be able to handle this case in Python 2. Right now stringify is just unicode in Python 2 which defaults to interpreting the input string as ascii encoded. If we did unicode(s, "utf-8") we'd still be able to encode strings like this.

➕ for still doing the error handling as the data might not necessarily be utf-8 or ascii (as seen in the flask case recently).

Does that mean that we'll push invalid data without having a way for the users to be noticed unless they dig into their logs?

Right @jd this would require someone to notice it in their logs. I don't think we have yet a better behavior to offer yet since this would be an error that we'd cause while tracing rather than at startup.

The way I understand it is that they are 2 cases:

The user triggers this problem: they did a programming error and need to fix it because they're sending the wrong data type to ddtrace.

We have a bug in an integration that needs to be fixed.

You have two choices:
a. Ignore (catch the exception and ignore/log it)
b. Fix it

This is picking a for 1 and 2 IIUC.

For 1. I don't think choosing for the user is a good experience. Once you pick a. Ignore there is no way for them to decide anything else. As a user, I dislike libraries hiding my bugs.

For 2. As a ddtrace maintainer, I'm not sure hiding integration issues in user's log is going to help us improve the user experience and quality of our integration

Thoughts?

jd · 2021-03-24T12:56:32Z

ddtrace/span.py

+        try:
+            self.meta[key] = stringify(value)
+        except UnicodeDecodeError:
+            log.warning("error setting tag %s, ignoring it", key, exc_info=True)


Hmmm this is again hiding error under the carpet, I'm not sure it's a good idea.

Could you provide more context about the motivation behind this change?

jd · 2021-03-24T13:08:40Z

tests/tracer/test_span.py

+def test_span_bad_unicode_set_tag():
+    span = Span(None, None)
+    span.set_tag("key", "🤔")
+    span._set_str_tag("key", "🤔")


Does that mean that we'll push invalid data without having a way for the users to be noticed unless they dig into their logs?

codecov-io · 2021-03-25T00:17:49Z

Codecov Report

Merging #2218 (03aa7d6) into master (081c25b) will increase coverage by 0.05%.
The diff coverage is 75.00%.

@@            Coverage Diff             @@
##           master    #2218      +/-   ##
==========================================
+ Coverage   91.53%   91.58%   +0.05%     
==========================================
  Files         570      570              
  Lines       40827    40834       +7     
==========================================
+ Hits        37371    37398      +27     
+ Misses       3456     3436      -20

Impacted Files	Coverage Δ
ddtrace/compat.py	`93.47% <ø> (ø)`
ddtrace/span.py	`90.80% <50.00%> (-0.70%)`	⬇️
tests/tracer/test_span.py	`98.54% <100.00%> (+0.01%)`	⬆️
ddtrace/profiling/exporter/http.py	`98.91% <0.00%> (-1.09%)`	⬇️
tests/commands/test_runner.py	`10.84% <0.00%> (+10.84%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 081c25b...03aa7d6. Read the comment docs.

jd · 2021-03-25T08:18:23Z

tests/tracer/test_span.py

+    span = Span(None, None)
+    span.set_tag("key", "🤔")
+    span._set_str_tag("key_str", "🤔")
+    # only span.set_tag() will fail


is there any reason not to use ensure_text also in set_tag?

We could though in that case we'd need to first get the string representation of the value object if its not a six.binary_type or six.text_type. I wanted to keep this PR to just _set_str_tag but if this works for this case (which is broken) I'd do the same for set_tag (which is less broken but still doesn't properly handle these binary and byte string cases).

I'm actually confused why set_tag would fail and _set_str_tag?

By "fail" I mean that an exception will be raised and handled in the case of set_tag:

>>> six.text_type("🤔") Traceback (most recent call last): File "<stdin>", line 1, in <module> UnicodeDecodeError: 'ascii' codec can't decode byte 0xf0 in position 0: ordinal not in range(128) >>> six.ensure_text("🤔") u'\U0001f914'

Oh yeah. I guess we really should be using ensure_text in set_tag. Using text_type sounds really like a bad idea. 😢

majorgreys · 2021-03-25T14:35:30Z

ddtrace/span.py

@@ -298,7 +299,10 @@ def set_tag(self, key, value=None):

    def _set_str_tag(self, key, value):
        # type: (str, str) -> None


Update type annotation

majorgreys · 2021-03-25T14:39:11Z

Also worth replacing ddtrace.compat.to_unicode with ensure_text.

majorgreys · 2021-03-25T14:42:25Z

ddtrace/span.py

@@ -298,7 +299,10 @@ def set_tag(self, key, value=None):

    def _set_str_tag(self, key, value):
        # type: (str, str) -> None
-        self.meta[key] = stringify(value)
+        try:
+            self.meta[key] = ensure_text(value)


Use errors="replace" so we don't fail for other non-unicode codecs eg #2155

jd · 2021-03-25T16:46:56Z

ddtrace/span.py

+        # type: (Text, AnyStr) -> None
+        try:
+            self.meta[key] = ensure_text(value, errors="replace")
+        except TypeError:


You should never get a TypeError there right?

Added a test for the case where this is called with something other than AnyStr, e.g. dict(a=1). But I admit this might be more defensive than we ought to be in an internal function which we are type annotating.

P403n1x87

just a reminder to wrap retries in str in the urllib3 integration when the retries is passed to trace_utils

…race-py into tag-unicode-decode-error

Kyle-Verhoog · 2021-03-29T19:56:20Z

ddtrace/span.py

@@ -310,7 +312,7 @@ def set_tag(self, key, value=None):

    def _set_str_tag(self, key, value):


Can we add to the docstring that errors in decoding will be replaced with ? as per https://docs.python.org/3/library/codecs.html#codecs.replace_errors

Kyle-Verhoog · 2021-03-29T19:58:00Z

tests/tracer/test_span.py

+    assert span.meta["key_str"] == u"🤔"
+
+
+@pytest.mark.skipif(sys.version_info.major >= 2, reason="This test does not apply to Python 2")


This test case will never run?

either the reason statement is wrong or the condition.

I suspect it was meant to be <= 2

Kyle-Verhoog · 2021-03-30T15:44:43Z

Looks like a bunch of grpc tests are failing due to an encoding issue 🤔

…race-py into tag-unicode-decode-error

Kyle-Verhoog · 2021-03-30T16:54:56Z

ddtrace/contrib/grpc/client_interceptor.py

@@ -102,7 +103,7 @@ def _handle_error(span, response_error, status_code):
            exc_val = to_unicode(response_error.details())
            span._set_str_tag(errors.ERROR_MSG, exc_val)
            span._set_str_tag(errors.ERROR_TYPE, status_code)


Should we stringify here too? Could the status code be an int?

(Also we should check all the usages of _set_str_tag to be sure that there aren't any other possibilities like this

status_code has already been coerced into unicode in this integration. Other integrations using _set_str_tag were already updated in this PR.

Kyle-Verhoog · 2021-04-01T19:57:43Z

@Mergifyio backport 0.48

* chore: correct typing hints for span.meta * fix: handle text types in _set_str_tag * chore: correct typing hints for span.meta * fix: handle text types in _set_str_tag * fix: handle text types in _set_str_tag * flake8 * add doc string, fix test * fix grpc traceback tag Co-authored-by: Brett Langdon <brett.langdon@datadoghq.com> Co-authored-by: Gabriele N. Tornetta <P403n1x87@users.noreply.github.com> (cherry picked from commit c19c334) # Conflicts: # ddtrace/contrib/trace_utils.py # ddtrace/span.py

mergify · 2021-04-01T19:58:32Z

Command backport 0.48: success

Backports have been created

#2264 fix: handle text types in _set_str_tag (bp #2218) has been created for branch 0.48

* fix typo in filter debug line (#2213) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * feat: support space separated DD_TAGS value (#2196) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add sync mode to AgentWriter (#2206) * feat(agent): support aws lambda extension * chore: fix flake8 issue with test * chore: add release note for datadog lambda extension * chore: fix import order * chore: fix flake8 issue * [serverless] fix broken test * remove unused import * separate out sync writer changes * add integration tests for sync writer Co-authored-by: DarcyRaynerDD <darcy.rayner@datadoghq.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * ci: run django test suite with ddtrace enabled (#2203) * Run django test suite with ddtrace enabled * Added pylibmc install to fix build error * Added django/ddtrace install commands to separate blocks * Fixed annotation * Installed generic version of ddtrace instead of checking out * Added documentation * Changed working directory, removed inherently failing test * Reverted working directory change * Reverted skipping specific test, added pythonpath * Added more docs Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * ci: run flask test suite with ddtrace enabled (#2190) * ci: run flask test suite with ddtrace enabled * Update .github/workflows/flask.yml Co-authored-by: Brett Langdon <me@brett.is> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Brett Langdon <brett.langdon@datadoghq.com> Co-authored-by: Brett Langdon <me@brett.is> Co-authored-by: Kyle Verhoog <kyle@verhoog.ca> * docs: add note about pip version requirement (#2220) * handle logwriter configure (#2219) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * test: move __init__ to utils (#2217) Having code in tests/__init__ makes ddtrace being imported before we have a chance to do anything, such as calling gevent.monkey.patch_all. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * refactor: distributed_tracing_enabled usages to trace_utils helper (#2216) * Modified trace util helper to account for inconsistencies in distributed_tracing config key, refactored usages * Refactored traceutils helper * Made int_config optional in the trace_util helper, updated affected integration files * Removed erroneous code addition to pyramid/trace.py * Make tracer tests less noisy (#2221) monkeypatch.setenv warns if the value given is not an instance of the base string type. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * refactor(worker): replace regular thread by PeriodicThread (#2205) Co-authored-by: Tahir H. Butt <tahir.butt@datadoghq.com> * Disable known failing Django tests (#2224) * Disable known failing Django tests * fix yaml multiline string Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * test(botocore): add test case for firehose args (#2227) This change adds a test case for #2204 to ensure that the params.Records argument is excluded from the span tags. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * refactor: move ddtrace.profiling._attr to ddtrace.utils (#2230) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Adding urllib3 integration (#1148) * draft of urllib3 integration * black formatting * set request tags on the span before the call is made in case of errors * rename span name and support retry tags based on CR feedback * add tests and fix a few small bugs * fix flake8 * add tests and circleci config * flake8 fixes * Update ddtrace/contrib/urllib3/__init__.py Co-Authored-By: Tahir H. Butt <tahir.butt@datadoghq.com> * use dd compat for urlparse and unparse * use SpanType constants for type of span use SpanTypes.HTTP * fix tests * fix flake8 from merge conflict resolution * adapt to latest changes on master * adopt latest standards for integrations Use Pin to add patch information to urllib3 requests. Also use riot for configuring the test environment * apply checklist points * wip: make tests pass * set tags on exception also fixed the tests to override HTTP configuration where required * docs(urllib3): add urllib3 integration docs * fix service name handling * fix copy-pasta errors and improve documentation * Update ddtrace/contrib/urllib3/__init__.py Co-authored-by: Kyle Verhoog <kyle@verhoog.ca> * Update riotfile.py Co-authored-by: Kyle Verhoog <kyle@verhoog.ca> * remove unused imports from test sources * use the new get_argument_value utility * Update tests/contrib/urllib3/test_urllib3.py Co-authored-by: Kyle Verhoog <kyle@verhoog.ca> * Update ddtrace/contrib/urllib3/patch.py Co-authored-by: Kyle Verhoog <kyle@verhoog.ca> * commit Tahir's suggestions * fix TracerTestCase import Co-authored-by: Tahir H. Butt <tahir.butt@datadoghq.com> Co-authored-by: Gabriele N. Tornetta <gabriele.tornetta@datadoghq.com> Co-authored-by: Gabriele N. Tornetta <P403n1x87@users.noreply.github.com> Co-authored-by: Kyle Verhoog <kyle@verhoog.ca> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Yun Kim <35776586+Yun-Kim@users.noreply.github.com> * Revert writer.write argument change (#2232) This is a breaking change which we're going to make at a later date. * Make processor private on Tracer (#2233) This isn't something we want to make public yet. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * fix(profiling): make sure PeriodicCollector start withing service lock (#2214) This makes sure the periodic collectors are started while the service lock is held to avoid race conditions while starting the services. This has no impact in practice since the Profiler class was made thread safe with #2179. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * refactor(forksafe): simplify code (#2222) This simplifies the code to use a banal decorator approach rather than a weird lambda-passed-as-argument-based 😉 It also adds typing and an `unregister` function to allow for… unregistering a function! Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * chore: correct typing hints for span.meta (#2237) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Lambda extension support (#2092) * feat(agent): support aws lambda extension * chore: fix flake8 issue with test * chore: add release note for datadog lambda extension * chore: fix import order * chore: fix flake8 issue * [serverless] fix broken test * remove unused import * Add docs, type hints Co-authored-by: Kyle Verhoog <kyle.verhoog@datadoghq.com> Co-authored-by: Kyle Verhoog <kyle@verhoog.ca> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Yun Kim <35776586+Yun-Kim@users.noreply.github.com> * fix(span): ensure ID args have the correct type (#2243) Among all the arguments of the Span constructor, the ones bearing IDs are the most important type-wise. This change adds checks to ensure that span_id, trace_id and parent_id all have the expected type. The rationale for adding the check at construction time is so that this error could be caught as soon as possible, rather than defer it to later processing stages, thus avoiding invalid entries in the processing queue. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * fix: only warn if using uwsgi with ddtrace-run (#2202) * fix: only warn if using uwsgi with ddtrace-run * remove DeprecationWarning prefix * update message Co-authored-by: Brett Langdon <brett.langdon@datadoghq.com> * Update ISSUE_TEMPLATE.md (#2248) * Use multiproject pipeline to deploy to reliability env (#2249) * Use multiproject pipeline for reliability env deploy * Use default branch Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * style(pymongo): format pymongo (#2250) * fix: handle text types in _set_str_tag (#2218) * chore: correct typing hints for span.meta * fix: handle text types in _set_str_tag * chore: correct typing hints for span.meta * fix: handle text types in _set_str_tag * fix: handle text types in _set_str_tag * flake8 * add doc string, fix test * fix grpc traceback tag Co-authored-by: Brett Langdon <brett.langdon@datadoghq.com> Co-authored-by: Gabriele N. Tornetta <P403n1x87@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Tahir H. Butt <tahir.butt@datadoghq.com> Co-authored-by: DarcyRaynerDD <darcy.rayner@datadoghq.com> Co-authored-by: Yun Kim <35776586+Yun-Kim@users.noreply.github.com> Co-authored-by: Julien Danjou <julien.danjou@datadoghq.com> Co-authored-by: Brett Langdon <brett.langdon@datadoghq.com> Co-authored-by: Brett Langdon <me@brett.is> Co-authored-by: Gabriele N. Tornetta <P403n1x87@users.noreply.github.com> Co-authored-by: Justin <justin.james.iso@gmail.com> Co-authored-by: Gabriele N. Tornetta <gabriele.tornetta@datadoghq.com> Co-authored-by: Darcy Rayner <50632605+DarcyRaynerDD@users.noreply.github.com> Co-authored-by: Laplie Anderson <randomanderson@users.noreply.github.com>

majorgreys requested a review from a team as a code owner March 23, 2021 21:59

Kyle-Verhoog reviewed Mar 23, 2021

View reviewed changes

jd reviewed Mar 24, 2021

View reviewed changes

majorgreys force-pushed the tag-unicode-decode-error branch from a8b2110 to 03aa7d6 Compare March 24, 2021 23:59

majorgreys force-pushed the tag-unicode-decode-error branch 2 times, most recently from 03ecbf6 to 2b68a35 Compare March 25, 2021 00:53

majorgreys changed the title ~~fix: ignore if UnicodeDecodeError with internal _set_str_tag~~ fix: handle PY2 text types in _set_str_tag Mar 25, 2021

majorgreys force-pushed the tag-unicode-decode-error branch from 2b68a35 to 21b51ad Compare March 25, 2021 01:28

majorgreys changed the title ~~fix: handle PY2 text types in _set_str_tag~~ fix: handle text types in _set_str_tag Mar 25, 2021

majorgreys added the changelog/no-changelog A changelog entry is not required for this PR. label Mar 25, 2021

majorgreys requested review from Kyle-Verhoog and jd March 25, 2021 01:36

majorgreys force-pushed the tag-unicode-decode-error branch 4 times, most recently from 3929d75 to 1c03804 Compare March 25, 2021 02:48

jd reviewed Mar 25, 2021

View reviewed changes

majorgreys commented Mar 25, 2021

View reviewed changes

majorgreys added this to the 0.48.0 milestone Mar 25, 2021

majorgreys force-pushed the tag-unicode-decode-error branch from 1c03804 to 48a7943 Compare March 25, 2021 16:41

jd reviewed Mar 25, 2021

View reviewed changes

majorgreys force-pushed the tag-unicode-decode-error branch from 48a7943 to 0ea5111 Compare March 25, 2021 16:49

P403n1x87 requested changes Mar 25, 2021

View reviewed changes

majorgreys force-pushed the tag-unicode-decode-error branch from 0ea5111 to cb2b4c8 Compare March 25, 2021 17:45

majorgreys requested a review from P403n1x87 March 25, 2021 17:49

P403n1x87 previously approved these changes Mar 25, 2021

View reviewed changes

majorgreys dismissed P403n1x87’s stale review via 283faed March 25, 2021 19:43

Merge branch 'tag-unicode-decode-error' of github.com:majorgreys/dd-t…

c6aa5ff

…race-py into tag-unicode-decode-error

Kyle-Verhoog reviewed Mar 29, 2021

View reviewed changes

add doc string, fix test

66e6d41

majorgreys force-pushed the tag-unicode-decode-error branch from e31138e to 66e6d41 Compare March 29, 2021 20:34

majorgreys requested review from brettlangdon and Kyle-Verhoog March 29, 2021 20:42

P403n1x87 previously approved these changes Mar 29, 2021

View reviewed changes

Kyle-Verhoog previously approved these changes Mar 29, 2021

View reviewed changes

Merge branch 'master' into tag-unicode-decode-error

6505398

jd previously approved these changes Mar 30, 2021

View reviewed changes

Merge branch 'master' into tag-unicode-decode-error

fad22e0

fix grpc traceback tag

da7a573

majorgreys dismissed stale reviews from jd, Kyle-Verhoog, and P403n1x87 via c7f19ef March 30, 2021 16:30

Merge branch 'tag-unicode-decode-error' of github.com:majorgreys/dd-t…

4d44299

…race-py into tag-unicode-decode-error

majorgreys force-pushed the tag-unicode-decode-error branch from c7f19ef to 4d44299 Compare March 30, 2021 16:31

Merge branch 'master' into tag-unicode-decode-error

d3de7a9

Kyle-Verhoog reviewed Mar 30, 2021

View reviewed changes

P403n1x87 approved these changes Mar 30, 2021

View reviewed changes

Kyle-Verhoog approved these changes Mar 30, 2021

View reviewed changes

mergify bot merged commit c19c334 into DataDog:master Mar 30, 2021

majorgreys deleted the tag-unicode-decode-error branch March 30, 2021 17:50

mergify bot mentioned this pull request Apr 1, 2021

fix: handle text types in _set_str_tag (bp #2218) #2264

Closed

plannigan mentioned this pull request Apr 29, 2021

grpc client fails when parse_method_path() does not return a value for package #2375

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: handle text types in _set_str_tag #2218

fix: handle text types in _set_str_tag #2218

majorgreys commented Mar 23, 2021 •

edited

Loading

Kyle-Verhoog Mar 23, 2021

jd Mar 24, 2021

majorgreys Mar 24, 2021

jd Mar 24, 2021

jd Mar 24, 2021

jd Mar 24, 2021

codecov-io commented Mar 25, 2021

jd Mar 25, 2021

majorgreys Mar 25, 2021

jd Mar 25, 2021

majorgreys Mar 25, 2021

jd Mar 26, 2021

majorgreys Mar 25, 2021

majorgreys commented Mar 25, 2021

majorgreys Mar 25, 2021

jd Mar 25, 2021

majorgreys Mar 25, 2021 •

edited

Loading

P403n1x87 left a comment

Kyle-Verhoog Mar 29, 2021

Kyle-Verhoog Mar 29, 2021

brettlangdon Mar 29, 2021

P403n1x87 Mar 29, 2021

Kyle-Verhoog commented Mar 30, 2021

Kyle-Verhoog Mar 30, 2021

Kyle-Verhoog Mar 30, 2021

majorgreys Mar 30, 2021

Kyle-Verhoog commented Apr 1, 2021

mergify bot commented Apr 1, 2021

		@@ -298,7 +299,10 @@ def set_tag(self, key, value=None):

		def _set_str_tag(self, key, value):
		# type: (str, str) -> None

		@@ -310,7 +312,7 @@ def set_tag(self, key, value=None):

		def _set_str_tag(self, key, value):

		assert span.meta["key_str"] == u"🤔"


		@pytest.mark.skipif(sys.version_info.major >= 2, reason="This test does not apply to Python 2")

fix: handle text types in _set_str_tag #2218

fix: handle text types in _set_str_tag #2218

Conversation

majorgreys commented Mar 23, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Mar 25, 2021

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

majorgreys commented Mar 25, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

majorgreys Mar 25, 2021 • edited Loading

Choose a reason for hiding this comment

P403n1x87 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kyle-Verhoog commented Mar 30, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kyle-Verhoog commented Apr 1, 2021

mergify bot commented Apr 1, 2021

majorgreys commented Mar 23, 2021 •

edited

Loading

majorgreys Mar 25, 2021 •

edited

Loading