Handle forward references

[justanr]

This is fixed on 3.6.5+ but we need to handle it properly - some sort of delayed field instantiation maybe.

• would delay only apply to forward references to the target it self

• if not, how to delay look up for types that don't have a scheme yet

[justanr]

Working for forward references to self -- need to figure out how to handle forward references to types that aren't scheme'd yet.

[justanr]

Thought: subclass Nested and make it registry/converter aware?

Pro: Circular schema now work.
Con: Stuff explodes very late; requires both schema to be generated off the same registry

[justanr]

Minimal reproducible example

from typing import List
from marshmallow_annotations import AnnotationSchema

class Foo:
    bars: List["Bar"]

class Bar:
    foos: List[Foo]


class FooSchema(AnnotationSchema):
    class Meta:
        target = Foo
        register_as_scheme = True

class BarSchema(AnnotationSchema):
    class Meta:
        target = Bar
        register_as_scheme = True

results in marshmallow_annotations.exceptions.AnnotationConversionError: No field factory found for <class '__main__.Bar'>

This can be worked around (in my actual case) by manually providing a nested field (the actual case I discovered this in was using sqlalchemy backrefs and providing a type hint on the backref'd type).

For an actual solution, not sure tbh. Could add a special marker entry in the Meta.Fields parser of something like "__DELAYED__": True (not married to the name, but the double underscores seems like a good idea as they're very unlikely to end up as parameter names) which would generate a converter that would reference the registry at dump/load time. This just means the AnnotationConversionError is delayed until first use if it's never registered :?

The thunk registered could be slightly smart and cache the converter after first usage. I'm really not sure what downstream effects there might be (e.g. if something is introspecting marshmallow fields to do something)

Something that I like less is adding some sort of reify function to the module that would need to be called before any usage of the actual schemes and would some how know "okay, these things had delayed registrations, let me go find them and then replace their thunks with actual reified fields and if I can't, I'll throw an exception" This would avoid the additional overhead of checking the registry at dump/load time but replaces it with mental overhead of using the library.

Essentially it's an eager version of using a thunk that replaces it's internal implementation with the actual field.

In this instance, if something is inspected marshmallow fields to gather information then that could be (in theory at least) delayed until after the reify function is called. Finding the "thunked" schemas might be as easy as getting a hold of the marshmallow schema registry, chaining all the values together and looking for any that carry like a __ANNOTATION_THUNKED_FIELD__ = True attribute (rather than doing something slower like looking over every field that exists on the classes):

def reify(*, schema_registry=None, converter_registry=None):
    if schema_registry is None:
        from marshmallow.class_registry import _registry as schema_registry

    if converter_registry is None:
        from marshmallow_annotations.registry import registry as converter_registry

    schemes = (s for s in chain.from_iterable(registry.values()) if getattr(s, "__ANNOATION_THUNKED_FIELD__", False))
    for schema in schemes:
        _reify_internal(schema, converter_registry)

def _reify_internal(schema, converter_registry):
    # find thunked fields
    # for each thunked field, generate actual converter field and replace thunk
    # remove __ANNOTATION_THUNKED_FIELD__
   pass

In either case, the thunk needs to carry over information of the hinted type as well as any options that should be passed to the converter.

And I don't think there's a reason both couldn't be supported.

[justanr]

Could possibly be handled completely in registry by returning a special ThunkedType field if the requested type isn't present already