Question: required Optional[Object] possible?

[gangefors]

• OS: debian 9
• Python version import sys; print(sys.version): 3.6.6
• Pydantic version import pydantic; print(pydantic.VERSION): 0.13

Based on my interpretation of the following documentation
the annotation only declaration tells pydantic that this field is required
and
The ellipsis ... just means “Required” same as annotation only declarations above.

and with this model

from typing import Optional

import pydantic


class Model(pydantic.BaseModel):
    str: str
    str_with_ellipse: str = ...
    str_with_default: str = 'c'
    str_or_none: Optional[str]
    str_or_none_with_ellipse: Optional[str] = ...
    str_or_none_with_default: Optional[str] = None

I would not expect the following result.

>>> import what
>>> what.Model(str='a', str_with_ellipse='b').dict()
{'str': 'a', 'str_with_ellipse': 'b', 'str_with_default': 'c', 'str_or_none': None, 'str_or_none_with_ellipse': None, 'str_or_none_with_default': None}

In the above example I can create a model without actually specifying the required attributes str_or_none and str_or_none_with_ellipse. Without me specifying any value, a default value of None is assigned to these attributes, contrary to what the documentation says.

The documentation have many places where None is set explicitly as the default value even if it's not needed by the current behaviour.

Is it intentional that Optional[] assigns None to the attribute even without me specifying a default value? And if so, what is the reasoning for this undocumented behaviour?

[gangefors]

Now that I've been digging around a bit in the issue list and open PRs I see that this might be related to #132 and #118, but that's just me guessing.

[samuelcolvin]

I think it's related to #132 but not quite the same.

The origin of this behaviour was my frustration with trafaret, where I constantly forgot to mark fields as both nullable and not required.

If you ask me should "nullable" imply "not required".

I agree that theoretically this is not always going to be what people require, but practically for purposes, it's fine.

The "zen" (perhaps a bad word to use around python atm. 😉) of pydantic (and perhaps this should be documented somewhere) is "the resulting modal will always take this form", not "the input data was correctly formed". In other words it is primarily a data parsing library not a data validation library.

All that said, I'd happily accept a PR to clarify that Optional implies not required.

[samuelcolvin]

no reply...

[skaaptjop]

I've just come across this now.
My use case is based on JSONAPI where one is required to supply a field with a data element
{"relationships": { "related": { "data": <xxx>} } }

Now xxx should a list of stuff, a single element of stuff or null. So the model would look like:

class Related(BaseModel):
   data: Union[List[ResId], ResId, None]

(forget about what ResId is for now, it's not important.

Point is, if one leaves 'data' out of the model then it has a very semantic different meaning to explicitly telling it that the data is null.
However, the default handling will set data to null.

Whether we're parsing or validating, we never explicitly told the model to use None as the default.
It just seems that it works differently for any type other than None which seems odd?

[samuelcolvin]

ye, None is absolutely a special case.

I can see your problem, however I think it would require a significant change to pydantic to change this behaviour.

I'm prepared to consider changing the behaviour for Optional to be required if no default is set, but at this stage without looking into it further I can't guarantee it'll be worth the effort.

Feel free to have a go at it, but I warn you:

1. it involves getting in the nasty bits of field.py
2. I might come along and rewrite it or refuse the change if it's too large or negatively effects performance.

[skaaptjop]

After I went and read through the related issues and the topic of parsing vs validation for pydantic I kinda get it now. My use case is a validation use case which is different. Issues like mine are very simply solved by moving the problem to a validation space.

[samuelcolvin]

Doesn't make sense for something with the word "optional" to be required, closing this.