load_default reusing property instance between loaded instances

[sebarys]

Hi

I've defined following schema for my HTTP request (use flask as http server)

class Nested:
    def __init__(self, msg: str, variables: dict[str, str]):
        self.msg = msg
        self.variables = variables


class NestedSchema(BaseSchema):
    msg = fields.String(required=True, validate=[validate.Length(min=1)])
    variables = \
        fields.Dict(keys=fields.Str(), values=fields.Str(), allow_none=True, required=False, load_default={})

    @post_load
    def make(self, data, **kwargs) -> Nested:
        return Nested(**data)

class RequestPayload:
    def __init__(self, nested: Nested, score: Optional[float]):
        self.nested = nested
        self.score = score


class RequestPayloadSchema(BaseSchema):
    nested = fields.Nested(NestedSchema, required=True, allow_none=False)
    score = fields.Float(allow_none=True,
                               validate=[validate.Range(min=0.0, min_inclusive=True, max=1.0, max_inclusive=True)],
                               required=False,
                               load_default=None
                               )

    @post_load
    def make(self, data, **kwargs) -> RequestPayload:
        return RequestPayload(**data)

When testing application locally I've noticed super strange behaviour: for subsequent HTTP requests when variables is not provided in request payload I sometimes see that it is not empty dict.

It looks that load_default re-use instance so when during internal processing/some request provide value for it, requests without this value won't have empty dict but contain value from different request.

Is it a way to make sure that default values will always be as defined (not share mutable instance)?

[lafrech]

This is the dreaded Python "mutable default argument" pitfall. Search here or on the Internet.

[sebarys]

@lafrech thanks for quick answer!

I know that Python have such odd behaviour, but for serialisation use case can't library under the hood prepare deepcopy for each time load_default is used?

I don't see at first glance any use cases when it is desired so maybe it would be good to have reasonable default behaviour?

[lafrech]

Up till now our stance has been "pass a callable". Most of the time, it's gonna be list or dict and lambda functions make for more complete cases. Apart from users being caught by it, it has never been such a blocker we felt the need to protect the user (at the cost of a performance impact, be it minimal).

This can always be reconsidered, though. But is it worth it?