Add model signature generation #1034

Bobronium · 2019-11-26T10:58:43Z

Change Summary

Generating model signature based on its fields

import inspect
from pydantic import BaseModel

class FooModel(BaseModel):
    id: int
    name: str = None
    description: str = 'Foo'

print(inspect.signature(FooModel))
'''
(*, id: int, name: str = None, description: str = 'Foo') -> None
'''

Respects custom-defined init signature:

import inspect
from pydantic import BaseModel

class MyModel(BaseModel):
    id: int
    info: str = 'Foo'

    def __init__(self, id: int = 1, *, bar: str, **data) -> None:
        super().__init__(id=id, bar=bar, **data)

print(inspect.signature(MyModel))
'''
(id: int = 1, *, bar: str, info: str = 'Foo') -> None
'''

Related issue number

Closes #1032, timothycrosley/hypothesis-auto#10
Relates to fastapi/fastapi#318

ToDo:

Handle overlays in custom __init__
Rewrite signature tests
Fix issues with cython
Replace/ignore invalid params names in fields (Generate actual signature for BaseModel.__init__ #1032 (comment))
Add something in docs?

Checklist

Unit tests for the changes exist
Tests pass on CI and coverage remains at 100%
Documentation reflects the changes where applicable
changes/<pull request or issue id>-<github username>.md file added describing change
(see changes/README.md for details)

codecov · 2019-11-26T11:26:15Z

Codecov Report

Merging #1034 into master will not change coverage by %.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1034   +/-   ##
=======================================
  Coverage   99.94%   99.94%           
=======================================
  Files          21       21           
  Lines        3699     3699           
  Branches      728      728           
=======================================
  Hits         3697     3697           
  Misses          1        1           
  Partials        1        1

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d8d8fc1...d8d8fc1. Read the comment docs.

AttributeError: attribute '__code__' of 'cython_function_or_method' objects is not writable Possible there's other attrs that cannot be set in cython, but can't check myself yet

samuelcolvin · 2019-11-26T16:40:16Z

pydantic/main.py

+        orig_posonlycount = orig_code.co_posonlyargcount if PY38 else 0
+        orig_argnames = orig_code.co_varnames[: orig_code.co_argcount + orig_code.co_kwonlyargcount + orig_posonlycount]
+        orig_argnames_set = set(orig_argnames)
+        fields_names = tuple([name for name in fields if name not in orig_argnames_set])


this might not maintain order of orig_argnames I believe.

It doesn’t do anything with orig_argnames, but excluding them from fields. This is actually done to maintain their order.

pydantic/main.py

pydantic/utils.py

samuelcolvin · 2019-11-26T16:42:28Z

tests/test_main.py

@@ -1022,3 +1023,72 @@ class NewModel(DerivedModel, something=2):
        something = 1

    assert NewModel.something == 2
+
+
+def test_init_signature(capsys):


new test file please.

samuelcolvin · 2019-11-26T16:44:43Z

tests/test_main.py

+        assert param.kind is kind
+        assert param.default == default
+
+    assert Model.__init__.__name__ == '__init__'


can we add

assert help(Model.__init__) == """ ... """

Or equivalent to provide an easy to read check?

I’ve done in first commit, but because of different formatting, tests on CI failed. There were inconsistency with spaces between arguments. Probably we can fix it by comparing strings without spaces

Or different tests for different versions of python.

samuelcolvin · 2019-11-26T16:46:05Z

pydantic/main.py

+            orig_init.__globals__,
+            name=orig_init.__name__,
+        )
+        fake_init.__doc__ = new_init.__doc__ = orig_init.__doc__


Would it be a good idea to add a helpful docstring to the real __init__ which by default appears in help(MyModel.__init__)?

It’s not bad idea for sure. What should be written there?

Not sure, something like:

Create a new model by parsing and validating input data from keyword arguments, a ValidationError will be raised if the input data cannot be parsed to for a valid model.

Or something.

Also adding info about original signature to docstring on generation

samuelcolvin · 2019-11-26T16:47:01Z

pydantic/main.py

+        orig_code = orig_init.__code__
+        # cannot modify cls.__init__ directly,
+        # as it shared between children classes, so we copy it
+        cls.__init__ = new_init = FunctionType(


could we move from here on into a separate function?

Yeah, good idea. Where do you think is the best place for it? main, utils?

Don't really mind, either typing or utils I guess.

pydantic/main.py

Moved tests to separate file Moved __init__ creation to utils

Co-Authored-By: Samuel Colvin <samcolvin@gmail.com>

pydantic/main.py

pydantic/utils.py

samuelcolvin · 2019-11-28T16:43:21Z

pydantic/utils.py

+
+    doc = orig_init.__doc__ or ''
+    real_signature = inspect.signature(orig_init)
+    new_doc = f'Signature is generated based on model fields. Real signature:\n    {real_signature}\n'


I'm not sure adding the real signature here is very helpful.

I think we should just use orig_init.__doc__? Perhaps with ...\n(signature auto generated from model fields) appended or something.

samuelcolvin · 2019-11-28T16:47:04Z

tests/test_init_signature.py

+    sig = signature(Model.__init__)
+    assert sig != signature(BaseModel.__init__)
+
+    params = sig.parameters


I think this would be much easier to read and debug with something like

assert {n: str(p) for n, p in s.parameters.items()} = { ... }

Yeah, thats' right. But I'd went with

assert [str(p) for p in sig.parameters.values()] == [ 'self', 'id: int = 1', 'bar=2', 'baz: Any', "name: str = 'John Doe'", '**data', ]

to maintain the order.

samuelcolvin · 2019-11-28T16:47:26Z

tests/test_init_signature.py

+    assert m.baz == 'Ok!'
+    assert m.name == 'John Doe'
+
+    sig = signature(MyModel.__init__)


Bobronium · 2019-11-28T19:42:22Z

~~Oops, probably merging that test was a bad idea...~~
Fixed

dmontagu · 2019-11-28T20:57:58Z

pydantic/utils.py

+            orig_code, orig_init.__globals__, closure=orig_init.__closure__, name=orig_init.__name__
+        )
+    else:
+        orig_init = getattr(current_init, '__origin_init__', current_init)


Does this fork not work if not compiled? Just wondering why we can't use the same logic in both cases (it's totally fine if we have to, I'm just trying to understand)

I guess it would work, but it also will add redundant call to a stack which may cause confusion on step into model creation while debugging and will be also a little overhead. Current behavior just will lead python right into original init and won’t add any redundant calls, so I decided to stick with it, when possible.

I see where you are coming from.

Possible counter-point: this also means that there could be bugs that show up in the compiled version and not the un-compiled version, or vice versa. I think it would be extremely annoying to figure out what was going wrong in a reported issue if it dependended on the compiled status. Obviously that's always a possibility, but the chance seems considerably higher given an explicit logic fork on that condition. Do you still think it makes sense to keep the fork given that consideration?

@samuelcolvin I'd also be interested to get your opinion here.

Good point. Personally I don't see a problem here, but this is surely needs to be considered.

My problem here is not just about different routes, but also about different tracebacks:

Running the following code:

from datetime import datetime from typing import List from pydantic import BaseModel class User(BaseModel): id: int name = 'John Doe' signup_ts: datetime = None friends: List[int] = [] user = User()

With pydantic not compiled I get:

Traceback (most recent call last): File "test.py", line 14, in <module> user = User() File "/home/samuel/code/pydantic/pydantic/main.py", line 293, in __init__ raise validation_error pydantic.error_wrappers.ValidationError: 1 validation error for User id field required (type=value_error.missing)

But when compiled I get:

Traceback (most recent call last): File "test.py", line 14, in <module> user = User() File "pydantic/utils.py", line 169, in pydantic.utils.generate_typed_init.__init__ orig_init(*args, **kwargs) File "pydantic/main.py", line 293, in pydantic.main.BaseModel.__init__ raise validation_error pydantic.error_wrappers.ValidationError: 1 validation error for User id field required (type=value_error.missing)

I think it's important these two tracebacks look the same and I think the traceback should not have the extra pydantic.utils.generate_typed_init.__init__... line.

I see three possible solutions to this:

Find a way to re arrange this function to avoid this extra step. maybe not possible.

Find another work around for this problem, e.g.:

just modify the docstring to detail the arguments

implement our own custom signature() method that provides a sensible signature for a model, without changing the actual model code

Manually customise the traceback so it's always the same, and perhaps even more simplified, e.g. just

Traceback (most recent call last): File "test.py", line 14, in <module> user = User() pydantic.error_wrappers.ValidationError: 1 validation error for User id field required (type=value_error.missing)

or something.

What do you think?

You should look here for reasons why in cpython we cannot achieve same traceback as in python:
https://stackoverflow.com/questions/59072875/how-to-clone-function-in-cython-i-getting-systemerror-unknown-opcode/59087135#59087135

Yes, i see that, I tried this myself I got something to work see 5230383 or the init-signature-sandbox branch. But I think that approach has other problems.

I think the best solution is our own utility function to generate the signature for an init function. That inspect.signature() would be the same as now, but pydantic.inspect.signature() (or whatever) would give the new improved signature. Would that work for the original use case?

I don't get how exactly pydantic.inspect.signature() would help us solve copying problem.

Also, it seems like your approach would break code like this:

class MyModelsBase(BaseModel): def __init__(self, **data): super().__init__(self, **data) print(f'Initialized {self} with {data}') class MyModel(MyModelsBase): a: int MyModel(a=a) # no output, as on MyModel creation '__init__' was not in namespace and was replaced

2. Find another work around for this problem, e.g.:

just modify the docstring to detail the arguments

implement our own custom signature() method that provides a sensible signature for a model, without changing the actual model code

Even if we modify docstring to detail the arguments, or implement our own argument, they still will be modified on the same __init__ of a base model, so every declared model will rewrite it and eventually it'll end up with the docstring and signature of the last model created.

I also thought about declaring __annotations__, __kwdefaults__, __wrapped__ and __signature__ as python property so they will give signatures of current models, but that doesn't seem possible too, because WhateverModel.__init__ would still be bound to BaseModel and I don't see any way how these properties could know what model's signature to give on call

dmontagu · 2019-11-29T00:34:08Z

pydantic/utils.py

+    orig_othernames = orig_code.co_varnames[orig_allargscount:]
+
+    orig_argnames_set = set(orig_argnames)  # just for fast lookups
+    valid_fields_args = tuple([name for name in fields if name not in orig_argnames_set and name.isidentifier()])


Does this generate the right signature if a field has an alias and doesn't have allow_population_by_field_name=True?

At this point, aliases are not considered at all.
Can you please give me example of such model and what signature should look like?

And thanks for that question, I found a bug in current implementation: models with fields declared in namespace would have this in signature:

__init__(__pydantic_self__, *, a=FieldInfo(default=1, alias='A', extra={}), **data: Any) -> None

Upd. Fixed

class A(BaseModel): a: int = Field(alias='b')

It looks like your generated signature will be (ignoring any annotations hints):

__init__(__pydantic_self__, a)

but if you call __init__(a=1) you'll get an error. I would think the signature *should * be __init__(__pydantic_self__, b) in this case. This is what I meant by aliases.

Also, note that if you are using the type annotation from the field, the generated signature will actually also be wrong for any type that is not idempotent. In particular, at least currently, if you have a SecretStr field, you have to pass a str to the __init__ function and not a SecretStr. I opened an issue to fix this for SecretStr, but there are other types for which this is also an issue.

(Admittedly, this is also a problem in the PyCharm plugin and, in strict mode, the mypy plugin too.)

Ok, now I see. Aliases should be considered.

As for SecretStr and other custom types: I don't see any problems here since these annotations will be available only at runtime and won't bother type-checkers. Until some code checks incoming types at runtime and don't know how to deal with pydantic, I don't think having pydantic specific types in signature would be a problem. For example, FastAPI understands such annotations just fine:

class GetUserFilters(BaseModel): name: Optional[constr(strip_whitespace=True, max_length=70)] email: Optional[SecretStr] @app.get('/users') def get_users(params=Depends(GetUserFilters)): ...

Upd.
Having thoughts about this, maybe these types could have special attribute for actual type of expected value, that would be added to the field. This also probably could help PyCharm plugin to reveal correct types of expected values.

I'll open separate issue where we can discuss that

I agree they won't bother type checkers, but it still feels bad for it to be generated "wrong". I think there is a tradeoff here between it faithfully representing the valid signature of the method (i.e., including a type annotation of Any or similar to allow for parsing), and providing useful information.

In general, I would prefer to err on the side of "correctness" (e.g., generating an __init__ signature with Any as the annotation), but from a practical perspective I'm okay with it being occasionally wrong in a minor way like this if it is substantially more useful (and preferred in practice).

pydantic/main.py

samuelcolvin · 2019-11-29T13:13:42Z

pydantic/utils.py

+            orig_code, orig_init.__globals__, closure=orig_init.__closure__, name=orig_init.__name__
+        )
+    else:
+        orig_init = getattr(current_init, '__origin_init__', current_init)


My problem here is not just about different routes, but also about different tracebacks:

Running the following code:

from datetime import datetime from typing import List from pydantic import BaseModel class User(BaseModel): id: int name = 'John Doe' signup_ts: datetime = None friends: List[int] = [] user = User()

With pydantic not compiled I get:

Traceback (most recent call last): File "test.py", line 14, in <module> user = User() File "/home/samuel/code/pydantic/pydantic/main.py", line 293, in __init__ raise validation_error pydantic.error_wrappers.ValidationError: 1 validation error for User id field required (type=value_error.missing)

But when compiled I get:

Traceback (most recent call last): File "test.py", line 14, in <module> user = User() File "pydantic/utils.py", line 169, in pydantic.utils.generate_typed_init.__init__ orig_init(*args, **kwargs) File "pydantic/main.py", line 293, in pydantic.main.BaseModel.__init__ raise validation_error pydantic.error_wrappers.ValidationError: 1 validation error for User id field required (type=value_error.missing)

I think it's important these two tracebacks look the same and I think the traceback should not have the extra pydantic.utils.generate_typed_init.__init__... line.

I see three possible solutions to this:

Find a way to re arrange this function to avoid this extra step. maybe not possible.

Find another work around for this problem, e.g.:

just modify the docstring to detail the arguments

implement our own custom signature() method that provides a sensible signature for a model, without changing the actual model code

Manually customise the traceback so it's always the same, and perhaps even more simplified, e.g. just

Traceback (most recent call last): File "test.py", line 14, in <module> user = User() pydantic.error_wrappers.ValidationError: 1 validation error for User id field required (type=value_error.missing)

or something.

What do you think?

Co-Authored-By: Samuel Colvin <samcolvin@gmail.com>

Zac-HD

Really looking forward to this - people have been requesting pydantic support in Hypothesis, and this PR will mean we get it for free 🎉

docs/usage/models.md

pydantic/utils.py

Co-Authored-By: Samuel Colvin <samcolvin@gmail.com>

Bobronium · 2020-02-12T20:14:52Z

docs/usage/models.md

+To be included in signature, field alias or name must be a valid python identifier. *pydantic* prefer aliases over names, but may use field name if alias is not suitable. 
+If any field's alias and name are both invalid identifiers, `**data` argument will be added.
+
+Also, `**data` argument is always present in signature if `Config.extra` is `Extra.allow`


Maybe should also mention this here?

Suggested change

Also, `**data` argument is always present in signature if `Config.extra` is `Extra.allow`

Also, `**data` argument is always present in signature if `Config.extra` is `Extra.allow`.

!!! note

Types in signature are same as declared in model annotations. They are not necessarily the ones that can be parsed. That may change once #1055 solved.

@samuelcolvin, after this one gets resolved, I guess, we're ready to go 🎉

I've rewritten the docs to improve the english, and incorporated this suggestion.

samuelcolvin · 2020-02-27T20:26:09Z

For me this is ready to merge if you agree @MrMrRobat?

Bobronium · 2020-02-27T20:29:57Z

@samuelcolvin, is there any reason for changes in af3dd4d? Added test_kwargs would pass even without explicit declaration of **data parameter in this commit

I think that it may lead to confusion in such cases as:

    class Model(BaseModel):
        a: float
        b: int = 2

        def __init__(self, a: float, b: int):
            super().__init__(a=a, b=b)

        class Config:
            extra = 'allow'

Even though Config.extra is 'allow', there is no space for **data in custom declared init, so I think signature should be real (a: float, b: int) -> None rather than (a: float, b: int = 2, **data: Any) -> None

Bobronium · 2020-02-27T20:33:43Z

pydantic/utils.py

@@ -149,7 +149,7 @@ def generate_model_signature(

    present_params = signature(init).parameters.values()
    merged_params: Dict[str, Parameter] = {}
-    var_kw = None
+    var_kw = Parameter('data', Parameter.VAR_KEYWORD, annotation=Any)


My thoughts on this: #1034 (comment)

Bobronium · 2020-02-27T20:35:28Z

tests/test_model_signature.py

@@ -57,3 +57,14 @@ def test_invalid_identifiers_signature():
    assert _equals(str(signature(model)), '(*, valid_identifier: int = 123, yeah: int = 0) -> None')
    model = create_model('Model', **{'123 invalid identifier!': 123, '!': Field(0, alias='yeah')})
    assert _equals(str(signature(model)), '(*, yeah: int = 0, **data: Any) -> None')
+
+
+def test_kwargs():


As I said in #1034 (comment), this test would already work without changes in this commit

samuelcolvin · 2020-02-27T20:43:13Z

Yes sorry, maybe you're right. Feel free to revert this change or I will.

Sorry to mess with your PR, just trying to get it merged as I know I'm slow to respond here

Bobronium · 2020-02-27T20:48:38Z

That's totally fine, I don't mind any edits from your side.
I'll revert just in a few minutes and will add test for case above

Zac-HD · 2020-02-27T21:18:44Z

Why not make it an error to specify both a custom init method and config for extras-allowed?

As far as I can see it's always redundant (just check for a VAR_KW param), and may be inconsistent (usually indicating a bug).

pydantic/utils.py

samuelcolvin · 2020-02-27T22:12:39Z

pydantic/utils.py

+                if allow_names and field_name.isidentifier():
+                    param_name = field_name
+                else:
+                    use_var_kw = True


coverage is being weird, but it looks like this line might not be coverd

Seems like it's covered in test_invalid_identifiers_signature, maybe coverage struggles to detect it.

if it's covered, don't worry.

pydantic/utils.py

Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bobronium added 2 commits November 26, 2019 13:44

Add right __init__ signature generation

fe5b2bb

Add params overlay support for custom __init__, enhance tests

64c1b4c

Change fake_ini creation to solve AttributeError in cython

1a3b8cd

AttributeError: attribute '__code__' of 'cython_function_or_method' objects is not writable Possible there's other attrs that cannot be set in cython, but can't check myself yet

samuelcolvin reviewed Nov 26, 2019

View reviewed changes

dmontagu reviewed Nov 27, 2019

View reviewed changes

pydantic/main.py Outdated Show resolved Hide resolved

dmontagu reviewed Nov 27, 2019

View reviewed changes

pydantic/main.py Outdated Show resolved Hide resolved

Bobronium and others added 5 commits November 28, 2019 01:24

Cython compatibility, add __signature__, ignore non-identifiers fields

65b8cd2

Moved tests to separate file Moved __init__ creation to utils

Remove # pragma: no cover

283273a

Co-Authored-By: Samuel Colvin <samcolvin@gmail.com>

Fixed adding signature to doc, some compatibility fixes

1b1f7f1

Changed None to tuple() for py3.8 compatibility

a6ee462

Finally (hope so) fix issues with closure. Add new docstring each time

93f14f0

samuelcolvin reviewed Nov 28, 2019

View reviewed changes

Merge branch 'master' into right-init-signature

a6ea614

samuelcolvin and others added 6 commits November 28, 2019 22:52

coverage for UndefinedType

e2edb95

Checking signature more simple and obvious way

073050b

Changed doc caption to simple message, used Undefined, merge conflicts

c69630d

Compatibility with py3.6

f13e0fe

Add changes/ file

42fa132

Ah, that space... Reformatted and ready to go!

6bc0dee

dmontagu reviewed Nov 28, 2019

View reviewed changes

dmontagu reviewed Nov 29, 2019

View reviewed changes

samuelcolvin reviewed Nov 29, 2019

View reviewed changes

Bobronium and others added 4 commits November 29, 2019 20:18

Update pydantic/main.py

6942e34

Co-Authored-By: Samuel Colvin <samcolvin@gmail.com>

Update pydantic/main.py

3e6e7c7

Co-Authored-By: Samuel Colvin <samcolvin@gmail.com>

Fix signature for properties declared as fields

fdea110

Sort imports

f28dea3

Bobronium mentioned this pull request Nov 30, 2019

Enhance pydantic types, add info about actual expected type of a value. #1055

Closed

dmontagu mentioned this pull request Dec 1, 2019

Generate actual signature for BaseModel.__init__ #1032

Closed

Zac-HD reviewed Feb 12, 2020

View reviewed changes

docs/usage/models.md Outdated Show resolved Hide resolved

pydantic/utils.py Outdated Show resolved Hide resolved

Bobronium and others added 6 commits February 12, 2020 11:56

Remove inspect imports from top level

27a0089

Update docs/examples/models_signature.py

8656fa0

Co-Authored-By: Samuel Colvin <samcolvin@gmail.com>

Add missing import

6503145

Remove notes from doc

9abdf6e

change hypothesis-auto mention to hypothesis

5f807fc

Use None as slice end instead of len()

761032d

Bobronium commented Feb 12, 2020

View reviewed changes

samuelcolvin added 2 commits February 27, 2020 15:06

tweak generate_model_signature

af3dd4d

improve docs

7009810

Bobronium commented Feb 27, 2020

View reviewed changes

Revert af3dd4d, add fields to custom init only if var_kw declared

d8d8fc1

Bobronium commented Feb 27, 2020

View reviewed changes

pydantic/utils.py Show resolved Hide resolved

samuelcolvin reviewed Feb 27, 2020

View reviewed changes

Bobronium commented Feb 27, 2020

View reviewed changes

pydantic/utils.py Show resolved Hide resolved

samuelcolvin merged commit 0f94861 into pydantic:master Feb 27, 2020

Zac-HD mentioned this pull request Apr 11, 2020

Inspect __signature__ attribute of classes in builds() HypothesisWorks/hypothesis#2388

Merged

This was referenced Apr 22, 2020

Model signature generation makes incorrect signature for models with extras allowed and data attribute in spec #1418

Closed

Incorrect signature for instances of models #1419

Closed

samuelcolvin mentioned this pull request Apr 30, 2020

Fix/model instance signature #1427

Closed

4 tasks

samuelcolvin mentioned this pull request Jun 28, 2020

Make hypothesis + pydantic work #1666

Closed

Zac-HD mentioned this pull request Sep 4, 2020

Fix/use signature type hints HypothesisWorks/hypothesis#2580

Merged

Bobronium mentioned this pull request Dec 6, 2021

Enhancing editor completion for BaseModel.__init__ #3475

Closed

alexdrydew pushed a commit to alexdrydew/pydantic that referenced this pull request Dec 23, 2023

Bump ahash from 0.8.3 to 0.8.4 (pydantic#1034)

ad1d384

Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

-Also, `**data` argument is always present in signature if `Config.extra` is `Extra.allow`
+Also, `**data` argument is always present in signature if `Config.extra` is `Extra.allow`.
+!!! note
+    Types in signature are same as declared in model annotations. They are not necessarily the ones that can be parsed. That may change once #1055 solved.

Uh oh!

Add model signature generation #1034

Add model signature generation #1034

Uh oh!

Conversation

Bobronium commented Nov 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Change Summary

Respects custom-defined init signature:

Related issue number

ToDo:

Checklist

Uh oh!

codecov bot commented Nov 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Bobronium commented Nov 28, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Bobronium Nov 28, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Bobronium commented Nov 26, 2019 •

edited

Loading

codecov bot commented Nov 26, 2019 •

edited

Loading

Bobronium commented Nov 28, 2019 •

edited

Loading

Bobronium Nov 28, 2019 •

edited

Loading

dmontagu Nov 28, 2019 •

edited

Loading

Bobronium Nov 28, 2019 •

edited

Loading

samuelcolvin Nov 29, 2019 •

edited

Loading

Bobronium Nov 29, 2019 •

edited

Loading

Bobronium Nov 29, 2019 •

edited

Loading

dmontagu Nov 29, 2019 •

edited

Loading