cython

[samuelcolvin]

Related issue number

#547

Checklist

• Unit tests for the changes exist
• Tests pass on CI and coverage remains at 100%
• Documentation reflects the changes where applicable
• setup travis to build manylinux binary
• HISTORY.rst has been updated
  • if this is the first change since a release, please add a new section
  • include the issue number or this pull request number #<number>
  • include your github username @<whomever>

[codecov]

Codecov Report

Merging #548 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master   #548   +/-   ##
=====================================
  Coverage     100%   100%           
=====================================
  Files          15     15           
  Lines        2453   2491   +38     
  Branches      488    497    +9     
=====================================
+ Hits         2453   2491   +38

[samuelcolvin]

as of 7ebf254 pydantic is almost 2x faster on the benchmarks compared to master.

[samuelcolvin]

waiting on pypa/cibuildwheel#139

[samuelcolvin]

Okay, for me this is ready to merge and release, (unless @dmontagu wants to work on #553?).

It's available for install via pip install pydantic==0.27a1, please install, test and give feedback here.

I'll leave it for a few days and then assume either it works or no one could be bothered to test it, and merge.

[tiangolo]

Awesome! 🎉

I just installed it and ran all the FastAPI tests, they all passed. ✔️

I installed it on Windows and it installs (the non-Cythonized) version correctly. I ran all the FastAPI tests and they passed (except ujson, which is Linux-only). ✔️

The output of installing with:

pip install pydantic==0.27a1

is:

Collecting pydantic==0.27a1
  Downloading https://files.pythonhosted.org/packages/d4/42/e99aa0a6e4b6ea1fbbf729ffa67615348d66c156c353639a8667562c58f6/pydantic-0.27a1.tar.gz (60kB)
Requirement already satisfied: dataclasses>=0.6 in c:\users\maria\.virtualenvs\fastapi-zgjruwkb\lib\site-packages (from pydantic==0.27a1) (0.6)
Building wheels for collected packages: pydantic
  Building wheel for pydantic (setup.py): started
  Building wheel for pydantic (setup.py): finished with status 'done'
  Stored in directory: C:\Users\maria\AppData\Local\Temp\pip-ephem-wheel-cache-z3eqj6kg\wheels\28\db\fd\c5f791b5ffae394421a3bf005afd0f0b9b1364c75ddce40e45
Successfully built pydantic
Installing collected packages: pydantic
Successfully installed pydantic-0.27a1

So, my use case (FastAPI), of keeping it compatible with Windows and macOS is covered 🎉

---

This is awesome! Great job @samuelcolvin 🎉 🚀 ! (and thanks @dmontagu for the help and input).

---

Now I just cross fingers we can #520 work 🤞

[samuelcolvin]

yes, I will look at #520 before releasing this in case anything big has to change here.

[samuelcolvin]

@tiangolo, just out of interest since you have a windows machine: could you try installing after installing cython (pip install cython) and see if:

1. it tries to compile?
2. it succeeds in compiling?
3. if so whether it runs ok

[tiangolo]

I'll try (I'm doing it).

But I'm doing it on my wife's Windows (double booted) machine, without any C-stuff building toolchain.

It's taking quite some time at least (in Windows sometimes it doesn't report logs of what's doing).

[samuelcolvin]

thanks, if it's a hassle don't worry, I was just curious.

[tiangolo]

I left it until now trying to install in Windows, after installing cython.

It just hanged at:

Collecting pydantic==0.27a1
  Downloading https://files.pythonhosted.org/packages/d4/42/e99aa0a6e4b6ea1fbbf729ffa67615348d66c156c353639a8667562c58f6/pydantic-0.27a1.tar.gz (60kB)

Didn't build, didn't error out, until I killed it.

At least we know, it needs the C-stuff toolchain, etc.

---

Again, not a priority since it does install in Windows, just not with Cython.

[Gr1N]

Amazing improvement!

[dmontagu]

@tiangolo is it possible to try a pip install with the -v flag and share where it hangs? (I'd do it myself but I don't have access to a windows machine -- is there an easy way to try this in the cloud?) I don't know if cython issues are typical on windows but it would seem unfortunate if trying to install the package alongside cython just results in a hang

[tiangolo]

@tiangolo is it possible to try a pip install with the -v flag and share where it hangs? (I'd do it myself but I don't have access to a windows machine -- is there an easy way to try this in the cloud?) I don't know if cython issues are typical on windows but it would seem unfortunate if trying to install the package alongside cython just results in a hang

I'll try once I get to steal the Windows laptop again.

It would be misfortunate indeed, but I don't think it's an easy scenario to get. For that to happen, someone would have to explicitly install cython and not have the environment available to compile stuff with Cython.

I guess if someone installed/needed Cython in his installation, he/she would probably have the tooling working for it.

Still, I'll try with that once I get a chance.

[dgasmith]

This is awesome! Just as a note for the Conda deploys. Building a binary for each platform is quite straightforward and I am happy to integrate those builds when a release comes out.

[sullivancolin]

Not sure if this is fixable or is simply a gotcha.

When testing pydantic in a virtualenv without Cython and using the IPython interpreter, I received the following error:

$ ipython
Python 3.7.3 (default, Mar 27 2019, 09:23:15) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pydantic                                                                                                                                                                                                              
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-1-45515a945e82> in <module>
----> 1 import pydantic

~/.virtualenvs/project-svfG-b5G/lib/python3.7/site-packages/pydantic/__init__.py in <module>
      1 # flake8: noqa
----> 2 from . import dataclasses
      3 from .class_validators import validator
      4 from .env_settings import BaseSettings
      5 from .error_wrappers import ValidationError

~/.virtualenvs/project-svfG-b5G/lib/python3.7/site-packages/pydantic/dataclasses.py in <module>
      6 from .errors import DataclassTypeError
      7 from .fields import Required
----> 8 from .main import create_model, validate_model
      9 from .utils import AnyType
     10 

~/.virtualenvs/project-svfG-b5G/lib/python3.7/site-packages/pydantic/main.py in <module>
     66     compiled: bool = False
     67 else:  # pragma: no cover
---> 68     compiled = cython.compiled
     69 
     70 

AttributeError: module 'cython' has no attribute 'compiled'

I think the import cython in the try/except/else located here succeeded because of the %%cython magic which was located ~/.ipython/cython/. In the standard python interpreter it works fine and cython is not importable. In IPython, import cython succeeds but does not have the .compiled attribute

Removing the directory ~/.ipython/cython/ fixes the problem, but the directory is recreated whenever The %%cython magic is used, even from a totally separate virtualenv

[dmontagu]

@sullivancolin I think #573 should fix this problem pretty safely; it would be great if you could try running that and see if there are any other places where this problem comes up?

[sullivancolin]

@sullivancolin I think #573 should fix this problem pretty safely; it would be great if you could try running that and see if there are any other places where this problem comes up?

I can confirm that this fixed the issue when Cython is not present and using IPython. Thanks!