Context when validating data

[samuelcolvin]

This is discussed at length in #1170 but I'm creating a new issue to make things clearer.

---

Pydantic needs a way of accessing "context" when validating data, serialising data, creating schema.

Might be used via MyModel.parse_obj(raw_data, context=my_context).

Use cases:

• dynamic choices - E.g. when choosing from a select based on a entities you have access to in a db, obviously both the validation and schema for the field should be dynamic - this is very common frustration for me
• dynamic parameters on fields, e.g. max and min
• cases like this
• and so many more...

Questions/concerns:

• adding context with parse_obj, but I don't see way way to add context with MyModel(**data) maybe that's fine
• how do we include context to field definitions? e.g. foobar: conint(gt=context['...']) won't work
• how will context work in sub-models?
• will this require breaking changes that need it to go into v2? I hope not.

Another option is to encourage one of the options from #1170 and/or make one of them easier.

Feedback on how this might work and when you would use it very welcome.

[simkimsia]

Is it better to split some of the concerns or use cases into separate issues/features?

Meaning to say maybe context will only solve a subset of the concerns here for now

[samuelcolvin]

no, this is one feature which needs one discussion.

[simkimsia]

Copy that @samuelcolvin

[anentropic]

personally right now I'm interested in #1575 use case (auto-adding an attribute to child model instances which links to the parent model instance) and don't care much about context-aware serialization

so this is a comment in support of any solution which makes the former possible

[michaelurban]

I'm migrating to pydantic from marshmallow but am blocked on this. Any movement?

I have a parent model that contains a list of submodels. I need to update and validate two of the submodel's values (path and project_file) based on the value of a user supplied base_path command line argument.

base_path is a property of neither the parent nor the child model. It's user supplied runtime context. Is there any way to pass a value from a parent to a child model at runtime?

@pydantic_dataclasses.dataclass
class ProjectFile:
    version: str = "v1"
    images: typing.List[ProjectItem] = dataclasses.field(default_factory=list)


@pydantic_dataclasses.dataclass
class ProjectItem:
    path: str = ""
    project_file: str = "DefaultFileName"

    **# !!! This is not a property of the item, it's runtime context !!! #**
    base_path: str = ""
    
    @pydantic.root_validator
    def validate_build_context_path(cls, values):
        base_path = pathlib.Path(values["base_path"]).absolute()
        path = base_path / values["path"]
        project_file = path / values["project_file"]

        if not path.exists():
            raise ValueError(f"Path does not exist: `{path}`")
        elif not path.is_dir():
            raise ValueError(f"Path must be a directory: `{path}`")

        if not project_file.exists():
            raise ValueError(f"Project file not found: `{project_file}`")
        if not project_file.is_file():
            raise ValueError(
                f"Project file must be a file: `{project_file}` is a directory"
            )

        values["path"] = str(path)
        values["project_file"] = str(project_file)

        return values

Output:

User supplied base_path is: ~/Projects/project/

$ cmd --path ~/Projects/project
ProjectFile Validation Errors:
  [thing1]: Project file not found: `/home/mike/Projects/project/thing1/MissingProjectFile`
  [thing2]: Project file not found: `/home/mike/Projects/project/thing2/AnotherMissingProjectFile`

[samuelcolvin]

No movement is far from trivial, but it's definitely something I want.

Your best bet for now might be to use contextvars.

It would be helpful if you could give feedback on my questions at the top of this page, or show us your intended use case.

[michaelurban]

Thanks for the quick reply. I've updated my comment with something similar to my use case.

I like the idea of one library being the right choice for env settings validation, YAML file validation, and API validation. Which, pydantic VERY nearly is. Unfortunately accessing context data is such a common requirement that I'd mix and match three libs (Marshmallow + Django Rest Framework + envparse) over using one where it wasn't an option.