After Migration initial import of Model object is 100 times slower

[ErnestoLoma]

Initial Checks

• I confirm that I'm using Pydantic V2

Description

I'm in the process of migrating from pydantic 1.10.8 to 2.2.1. Using pydantic 1.10.8 an import of a model class took approximately 2.5 seconds. Using pydantic 2.2.1 it take approximately 244.2 seconds.
pydantic 1.10.8 output: The time to import Referenceable is: 2.5116748809814453 seconds
pydantic 2.2.1 output: The time to import Referenceable is: 244.24540090560913 seconds
Below is the script I am using for timing:

Example Code

import time

start = time.time()
from pyatlan.model.assets.asset00 import Referenceable
end = time.time()
print("The time to import Referenceable is:", (end-start), "seconds")

Python, Pydantic & OS Version

warnings.warn(f'`{import_path}` has been moved to `{new_location}`.')
             pydantic version: 2.2.1
        pydantic-core version: 2.6.1
          pydantic-core build: profile=release pgo=false
                 install path: /Users/ernesthill/projects/pydantic2/venv/lib/python3.9/site-packages/pydantic
               python version: 3.9.12 (main, May 11 2022, 15:24:03)  [Clang 13.1.6 (clang-1316.0.21.2.3)]
                     platform: macOS-13.4.1-arm64-arm-64bit
     optional deps. installed: ['typing-extensions']

Selected Assignee: @Kludex

[adriangb]

Seems related to #6768. Could you provide any sort of hints as to the structure of your model (recursiveness, number of fields, etc.), or anything else that might help us narrow down the issue?

[ErnestoLoma]

It's a very large model with a couple of circular dependencies. The source for the model I am importing is available here: Referenceable

[michaelgmiller1]

We are also experiencing very slow (unusable) import times for our similarly recursive model. Would be great to get some kind of fix, guessing caching for recursive models.

[adriangb]

Please see #6768 (comment)

[adriangb]

It's a very large model with a couple of circular dependencies. The source for the model I am importing is available here: Referenceable

I get a 404 now. @ErnestoLoma would you mind testing out main or the next release to see if it helps your use case?

[ErnestoLoma]

@adriangb Adrian I tried it with pydantic 2.3.0
pydantic version: 2.3.0
pydantic-core version: 2.6.3
pydantic-core build: profile=release pgo=false
install path: /usr/local/lib/python3.9/site-packages/pydantic
python version: 3.9.16 (main, May 23 2023, 09:48:30) [GCC 8.3.0]
platform: Linux-5.15.49-linuxkit-pr-aarch64-with-glibc2.28
optional deps. installed: ['typing-extensions']
The time to import was 280.74444603919983 seconds. I did the test in a docker container based on python:3.9.16-slim-buster. I did pip install of my requirements.txt file which includes the following:

• requests>=2.31
• pydantic==2.3.0
• jinja2==3.1.2
  I alse repeated the test setting the environment variable PYDANTIC_SKIP_VALIDATING_CORE_SCHEMAS=1. The time to import was 307.6565330028534 seconds. The code for the test is in the file try_pyndantic2.py.
  You said you got a 404. Could you tell me what code you ran and what version of python you used?

[adriangb]

2.3.0 does not have the fix. Please try 2.4.0 or main.

[ErnestoLoma]

@adriangb The time to import was 214.18017506599426 seconds. So slightly better.
pydantic version: 2.4.0
pydantic-core version: 2.10.0
pydantic-core build: profile=release pgo=false
install path: /Users/ernesthill/projects/pydantic2/venv/lib/python3.9/site-packages/pydantic
python version: 3.9.12 (main, May 11 2022, 15:24:03) [Clang 13.1.6 (clang-1316.0.21.2.3)]
platform: macOS-13.5.2-arm64-arm-64bit
related packages: mypy-1.5.1 typing_extensions-4.7.1 pydantic-settings-2.0.3

I did the install using pip install git+https://github.com/pydantic/pydantic.git

[sydney-runkle]

@ErnestoLoma,

Could you try against main now?

[michaelgmiller1]

Is there any progress on this issue? It is a big blocker for us to migrate to v2.

[sydney-runkle]

@michaelgmiller1,

We'll look to make some progress on this issue in the coming weeks, as we know it's limiting folks in terms of ability to upgrade!