Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JavaScript negative timestamp are not supported #1600

Closed
mlbiche opened this issue Jun 4, 2020 · 5 comments · Fixed by #1605
Closed

JavaScript negative timestamp are not supported #1600

mlbiche opened this issue Jun 4, 2020 · 5 comments · Fixed by #1605
Labels

Comments

@mlbiche
Copy link
Contributor

@mlbiche mlbiche commented Jun 4, 2020

Bug

When parsing a JavaScript positive timestamp (number of milliseconds since EPOCH), pydantic parses it properly. However, for a negative JavaScript timestamp (before 1st January 1970), a date value out of range error is thrown.

  File "pydantic/main.py", line 447, in pydantic.main.BaseModel.parse_obj
  File "pydantic/main.py", line 336, in pydantic.main.BaseModel.__init__
  File "pydantic/main.py", line 887, in pydantic.main.validate_model
  File "pydantic/fields.py", line 549, in pydantic.fields.ModelField.validate
  File "pydantic/fields.py", line 704, in pydantic.fields.ModelField._validate_singleton
  File "pydantic/fields.py", line 711, in pydantic.fields.ModelField._apply_validators
  File "pydantic/class_validators.py", line 313, in pydantic.class_validators._generic_validator_basic.lambda11
  File "pydantic/datetime_parse.py", line 169, in pydantic.datetime_parse.parse_datetime
  File "pydantic/datetime_parse.py", line 80, in pydantic.datetime_parse.from_unix_seconds
OverflowError: date value out of range

Version

Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":

             pydantic version: 1.5.1
            pydantic compiled: True
                 install path: /Users/thibaud/Documents/Programmation/Scopyleft/Terrapeutes/aposto-server/venv/lib/python3.7/site-packages/pydantic
               python version: 3.7.3 (default, Mar  6 2020, 22:34:30)  [Clang 11.0.3 (clang-1103.0.32.29)]
                     platform: Darwin-19.5.0-x86_64-i386-64bit
     optional deps. installed: ['typing-extensions', 'email-validator']

Code example

The following code raises the describe error (while it would correctly work with { "jsTimestamp": 1118102400000 }).

from datetime import datetime

from pydantic import BaseModel


class MyModel(BaseModel):
    jsTimestamp: datetime

MyModel.parse_obj({ "jsTimestamp": -1118102400000 })

Possible solution

A solution would be to update the from_unix_seconds function in datetime_parse.py to compare seconds with MS_WATERSHED as absolute value.

@mlbiche mlbiche added the bug label Jun 4, 2020
@samuelcolvin
Copy link
Owner

@samuelcolvin samuelcolvin commented Jun 4, 2020

Happy to accept a PR to fix this.

I guess we should:

  • raise the correct error resulting in a validation error if the datetime is out of bounds
  • accept negative timestamps if they result in valid datetimes - there's nothing implicitly wrong with the unix timestamp of -1. Stuff happened before 1970 apparently.
  • comparing with MS_WATERSHED as an absolute number makes sense I suppose, we would just need to update the docs to be explicit.

@mlbiche
Copy link
Contributor Author

@mlbiche mlbiche commented Jun 5, 2020

I don't get this :

* raise the correct error resulting in a validation error if the datetime is out of bounds

For the rest, everything is running fine and ready for PR

@samuelcolvin
Copy link
Owner

@samuelcolvin samuelcolvin commented Jun 5, 2020

I think it might be fine, I just meant:

make sure passing timestamps like 1_000_000_000_000_000 or -100_000_000_000_000 cause sensible validation errors not other exceptions.

@mlbiche
Copy link
Contributor Author

@mlbiche mlbiche commented Jun 5, 2020

Yes, it runs fine with these timestamps. The generated datetime does not match with the real date as these timestamps are greater than MS_WATERSHED but it is parsed without any exception.

@samuelcolvin
Copy link
Owner

@samuelcolvin samuelcolvin commented Jun 5, 2020

Best to submit a PR and we can discuss from there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants