[refactor] Add BuildStep, remove AttributeBuilder

- Rename pre/post declarations
- A BuildStep holds the whole building context
- Centralize declaration parsing logic

ERROR: Changed argument names in PostGeneration (public API)
WARNING: Renamed _meta.pre_declarations/post_declarations
WARNING: No longer possible to override a RelatedFactory value from a
        passed-in param (failing test, but no docs)

docs/changelog.rst

@@ -292,8 +292,8 @@ This takes care of all ``FACTORY_FOR`` occurences; the files containing other at
 
     - Add :class:`~factory.fuzzy.FuzzyDate` thanks to `saulshanabrook <https://github.com/saulshanabrook>`_
     - Add :class:`~factory.fuzzy.FuzzyDateTime` and :class:`~factory.fuzzy.FuzzyNaiveDateTime`.
-    - Add a :attr:`~factory.containers.LazyStub.factory_parent` attribute to the
-      :class:`~factory.containers.LazyStub` passed to :class:`~factory.LazyAttribute`, in order to access
+    - Add a :attr:`~factory.builder.Resolver.factory_parent` attribute to the
+      :class:`~factory.builder.Resolver` passed to :class:`~factory.LazyAttribute`, in order to access
       fields defined in wrapping factories.
     - Move :class:`~factory.django.DjangoModelFactory` and :class:`~factory.mogo.MogoFactory`
       to their own modules (:mod:`factory.django` and :mod:`factory.mogo`)

docs/internals.rst

@@ -1,2 +1,90 @@
 Internals
 =========
+
+.. currentmodule:: factory
+
+Behind the scenes: steps performed when parsing a factory declaration, and when calling it.
+
+
+This section will be based on the following factory declaration:
+
+.. literalinclude:: ../tests/test_docs_internals.py
+    :pyobject: UserFactory
+
+
+Parsing, Step 1: Metaclass and type declaration
+-----------------------------------------------
+
+1. Python parses the declaration and calls (thanks to the metaclass declaration):
+
+   .. code-block:: python
+
+        factory.base.BaseFactory.__new__(
+            'UserFactory',
+            (factory.Factory,),
+            attributes,
+        )
+
+2. That metaclass removes :attr:`~Factory.Meta` and :attr:`~Factory.Params` from the class attributes,
+   then generate the actual factory class (according to standard Python rules)
+3. It initializes a :class:`FactoryOptions` object, and links it to the class
+
+
+Parsing, Step 2: adapting the class definition
+-----------------------------------------------
+
+1. The :class:`FactoryOptions` reads the options from the :attr:`class Meta <Factory.Meta>` declaration
+2. It finds a few specific pointer (loading the model class, finding the reference
+   factory for the sequence counter, etc.)
+3. It copies declarations and parameters from parent classes
+4. It scans current class attributes (from ``vars()``) to detect pre/post declarations
+5. Declarations are split among pre-declarations and post-declarations
+   (a raw value shadowing a post-declaration is seen as a post-declaration)
+
+
+.. note:: A declaration for ``foo__bar`` will be converted into parameter ``bar``
+          for declaration ``foo``.
+
+
+Instantiating, Step 1: Converging entrypoints
+---------------------------------------------
+
+First, decide the strategy:
+
+- If the entrypoint is specific to a strategy (:meth:`~Factory.build`,
+  :meth:`~Factory.create_batch`, ...), use it
+- If it is generic (:meth:`~Factory.generate`, :meth:`Factory.__call__`),
+  use the strategy defined at the :attr:`class Meta <Factory.Meta>` level
+
+
+Then, we'll pass the strategy and passed-in overrides to the :meth:`~Factory._generate` method.
+
+.. note:: According to the project roadmap, a future version will use a :meth:`~Factory._generate_batch`` at its core instead.
+
+A factory's :meth:`~Factory._generate` function actually delegates to a ``StepBuilder()`` object.
+This object will carry the overall "build an object" context (strategy, depth, and possibly other).
+
+
+Instantiating, Step 2: Preparing values
+---------------------------------------
+
+1. The ``StepBuilder`` merges overrides with the class-level declarations
+2. The sequence counter for this instance is initialized
+3. A ``Resolver`` is set up with all those declarations, and parses them in order;
+   it will call each value's ``evaluate()`` method, including extra parameters.
+4. If needed, the ``Resolver`` might recurse (through the ``StepBuilder``, e.g when
+   encountering a :class:`SubFactory`.
+
+
+Instantiating, Step 3: Building the object
+------------------------------------------
+
+1. The ``StepBuilder`` fetches the attributes computed by the ``Resolver``.
+2. It applies renaming/adjustment rules
+3. It passes them to the :meth:`FactoryOptions.instantiate` method, which
+   forwards to the proper methods.
+4. Post-declaration are applied (in declaration order)
+
+
+.. note:: This document discusses implementation details; there is no guarantee that the
+          described methods names and signatures will be kept as is.

docs/reference.rst

@@ -804,7 +804,7 @@ accept the object being built as sole argument, and return a value.
 
 
 The object passed to :class:`LazyAttribute` is not an instance of the target class,
-but instead a :class:`~containers.LazyStub`: a temporary container that computes
+but instead a :class:`~builder.Resolver`: a temporary container that computes
 the value of all declared fields.
 
 
@@ -1273,7 +1273,7 @@ Obviously, this "follow parents" ability also handles overriding some attributes
 
 
 This feature is also available to :class:`LazyAttribute` and :class:`LazyAttributeSequence`,
-through the :attr:`~containers.LazyStub.factory_parent` attribute of the passed-in object:
+through the :attr:`~builder.Resolver.factory_parent` attribute of the passed-in object:
 
 .. code-block:: python
 

factory/base.py

@@ -5,8 +5,9 @@
 
 import collections
 import logging
+import warnings
 
-from . import containers
+from . import builder
 from . import declarations
 from . import errors
 from . import utils
@@ -148,21 +149,21 @@ class FactoryOptions(object):
     def __init__(self):
         self.factory = None
         self.base_factory = None
-        self.declarations = {}
-        self.postgen_declarations = {}
+        self.base_declarations = {}
         self.parameters = {}
         self.parameters_dependencies = {}
+        self.pre_declarations = builder.DeclarationSet()
+        self.post_declarations = builder.DeclarationSet()
 
         self._counter = None
         self.counter_reference = None
 
     @property
-    def sorted_postgen_declarations(self):
-        """Get sorted postgen declaration items."""
-        return sorted(
-            self.postgen_declarations.items(),
-            key=lambda item: item[1].creation_counter,
-        )
+    def declarations(self):
+        base_declarations = dict(self.base_declarations)
+        for name, param in self.parameters.items():
+            base_declarations.update(param.as_declarations(name, base_declarations))
+        return base_declarations
 
     def _build_default_options(self):
         """"Provide the default value for all allowed fields.
@@ -215,18 +216,17 @@ def contribute_to_class(self, factory, meta=None, base_meta=None, base_factory=N
 
         self.counter_reference = self._get_counter_reference()
 
+        # Scan the inheritance chain, starting from the furthest point,
+        # excluding the current class, to retrieve all declarations.
         for parent in reversed(self.factory.__mro__[1:]):
             if not hasattr(parent, '_meta'):
                 continue
-            self.declarations.update(parent._meta.declarations)
-            self.postgen_declarations.update(parent._meta.postgen_declarations)
+            self.base_declarations.update(parent._meta.base_declarations)
             self.parameters.update(parent._meta.parameters)
 
         for k, v in vars(self.factory).items():
-            if self._is_declaration(k, v):
-                self.declarations[k] = v
-            if self._is_postgen_declaration(k, v):
-                self.postgen_declarations[k] = v
+            if self._is_declaration(k, v) or self._is_postgen_declaration(k, v):
+                self.base_declarations[k] = v
 
         if params is not None:
             for k, v in vars(params).items():
@@ -235,6 +235,7 @@ def contribute_to_class(self, factory, meta=None, base_meta=None, base_factory=N
 
         self._check_parameter_dependencies(self.parameters)
 
+        self.pre_declarations, self.post_declarations = builder.parse_declarations(self.declarations)
 
     def _get_counter_reference(self):
         """Identify which factory should be used for a shared counter."""
@@ -309,21 +310,21 @@ def prepare_arguments(self, attributes):
 
         return args, kwargs
 
-    def instantiate(self, strategy, args, kwargs):
+    def instantiate(self, step, args, kwargs):
         model = self.get_model_class()
 
-        if strategy == BUILD_STRATEGY:
+        if step.builder.strategy == BUILD_STRATEGY:
             return self.factory._build(model, *args, **kwargs)
-        elif strategy == CREATE_STRATEGY:
+        elif step.builder.strategy == CREATE_STRATEGY:
             return self.factory._create(model, *args, **kwargs)
         else:
-            assert strategy == STUB_STRATEGY
+            assert step.builder.strategy == STUB_STRATEGY
             return StubObject(**kwargs)
 
-    def use_postgeneration_results(self, create, instance, results):
+    def use_postgeneration_results(self, step, instance, results):
         self.factory._after_postgeneration(
             instance=instance,
-            create=create,
+            step=step,
             results=results,
         )
 
@@ -459,20 +460,15 @@ def attributes(cls, create=False, extra=None):
             applicable; the current list of computed attributes is available
             to the currently processed object.
         """
-        force_sequence = None
-        if extra:
-            force_sequence = extra.pop('__sequence', None)
-        log_ctx = '%s.%s' % (cls.__module__, cls.__name__)
-        logger.debug(
-            "BaseFactory: Preparing %s.%s(extra=%s)",
-            cls.__module__,
-            cls.__name__,
-            utils.log_repr(extra),
-        )
-        return containers.AttributeBuilder(cls, extra, log_ctx=log_ctx).build(
-            create=create,
-            force_sequence=force_sequence,
+        warnings.warn(
+            "Usage of Factory.attributes() is deprecated.",
+            DeprecationWarning,
+            stacklevel=2,
         )
+        declarations = cls._meta.pre_declarations.as_dict()
+        declarations.update(extra or {})
+        from . import helpers
+        return helpers.make_factory(dict, **declarations)
 
     @classmethod
     def declarations(cls, extra_defs=None):
@@ -482,7 +478,12 @@ def declarations(cls, extra_defs=None):
             extra_defs (dict): additional definitions to insert into the
                 retrieved DeclarationDict.
         """
-        decls = cls._meta.declarations.copy()
+        warnings.warn(
+            "Factory.declarations is deprecated; use Factory._meta.pre_declarations instead.",
+            DeprecationWarning,
+            stacklevel=2,
+        )
+        decls = cls._meta.pre_declarations.as_dict()
         decls.update(extra_defs or {})
         return decls
 
@@ -505,31 +506,11 @@ def _generate(cls, strategy, params):
                 "Ensure %(f)s.Meta.model is set and %(f)s.Meta.abstract "
                 "is either not set or False." % dict(f=cls.__name__))
 
-        create = bool(strategy == CREATE_STRATEGY)
-
-        attrs = cls.attributes(create=create, extra=params)
-        # Extract declarations used for post-generation
-        postgen_attributes = {}
-
-        for name, decl in cls._meta.sorted_postgen_declarations:
-            postgen_attributes[name] = decl.extract(name, attrs)
-
-        # Generate the object
-        args, kwargs = cls._meta.prepare_arguments(attrs)
-        obj = cls._meta.instantiate(strategy, args, kwargs)
-
-        # Handle post-generation attributes
-        results = {}
-        for name, decl in cls._meta.sorted_postgen_declarations:
-            extraction_context = postgen_attributes[name]
-            results[name] = decl.call(obj, create, extraction_context)
-
-        cls._meta.use_postgeneration_results(create, obj, results)
-
-        return obj
+        step = builder.StepBuilder(cls._meta, params, strategy)
+        return step.build()
 
     @classmethod
-    def _after_postgeneration(cls, instance, create, results=None):
+    def _after_postgeneration(cls, instance, step, results=None):
         """Hook called after post-generation declarations have been handled.
 
         Args: