TypeError: BaseModel.validate() takes 2 positional arguments but 3 were given

[radiatorius1024]

An error is received when executing flow in Prefect 2.13.7...2.13.8:

from prefect_airbyte.server import AirbyteServer
from prefect_airbyte.connections import AirbyteConnection
from prefect_airbyte.flows import run_connection_sync

server = AirbyteServer(server_host="192.168.1.126", server_port=8000)

connection = AirbyteConnection(
airbyte_server=server,
connection_id="b35180dc-6bd1-48b0-b0cd-b99deb6caaaa",
status_updates=True
)

@flow
def extract_data_subflow():
run_connection_sync(airbyte_connection=connection)

pydantic_self.pydantic_validator.validate_python(data, self_instance=pydantic_self)
TypeError: BaseModel.validate() takes 2 positional arguments but 3 were given

[chrisguidry]

Hi @radiatorius1024, which version of prefect-airbyte are you running (pip freeze | grep prefect-airbyte)?

With prefect==2.13.7, we've removed the version constraint on pydantic, which allows pydantic>=2 to be installed, but that may be incompatible with older versions of prefect-airbyte. In prefect-airbyte==0.3.0, we've made the import adjustments to allow you to have either version of pydantic installed. If you aren't on 0.3.0, can you bump your version and let us know if that works?

[radiatorius1024]

Hi. Yes I have heard about the changes so the latest version is used:
pip freeze | grep prefect-airbyte -> prefect-airbyte==0.3.0

[chrisguidry]

Thanks @radiatorius1024, I should have also asked: what version of pydantic do you have installed as well?

[radiatorius1024]

Here are all installed libraries (Prefect installed on Docker, but same error with local installation):
Python 3.11.6

pip freeze

aiobotocore==2.7.0
aiohttp==3.8.6
aioitertools==0.11.0
aiosignal==1.3.1
aiosqlite==0.19.0
alembic==1.12.0
annotated-types==0.6.0
anyio==3.7.1
apprise==1.6.0
asgi-lifespan==2.1.0
async-timeout==4.0.3
asyncpg==0.28.0
attrs==23.1.0
botocore==1.31.64
cachetools==5.3.1
certifi==2023.7.22
cffi==1.16.0
charset-normalizer==3.3.0
click==8.1.7
cloudpickle==3.0.0
colorama==0.4.6
coolname==2.2.0
croniter==1.4.1
cryptography==41.0.4
dask==2023.10.0
dateparser==1.1.8
distributed==2023.10.0
docker==6.1.3
frozenlist==1.4.0
fsspec==2023.10.0
google-auth==2.23.3
graphviz==0.20.1
greenlet==3.0.0
griffe==0.36.7
h11==0.14.0
h2==4.1.0
hpack==4.0.0
httpcore==0.18.0
httpx==0.25.0
hyperframe==6.0.1
idna==3.4
importlib-metadata==6.8.0
Jinja2==3.1.2
jmespath==1.0.1
jsonpatch==1.33
jsonpointer==2.4
jsonschema==4.19.1
jsonschema-specifications==2023.7.1
kubernetes==28.1.0
locket==1.0.0
Mako==1.2.4
Markdown==3.5
markdown-it-py==3.0.0
MarkupSafe==2.1.3
mdurl==0.1.2
msgpack==1.0.7
multidict==6.0.4
numpy==1.26.1
oauthlib==3.2.2
orjson==3.9.9
packaging==23.2
pandas==2.1.1
partd==1.4.1
pathspec==0.11.2
pendulum==2.1.2
prefect @ file:///opt/prefect/dist/prefect.tar.gz
prefect-airbyte==0.3.0
prefect-dask==0.2.5
psutil==5.9.6
pyasn1==0.5.0
pyasn1-modules==0.3.0
pycparser==2.21
pydantic==2.4.2
pydantic_core==2.10.1
Pygments==2.16.1
pyodbc==5.0.1
python-dateutil==2.8.2
python-slugify==8.0.1
pytz==2023.3.post1
pytzdata==2020.1
PyYAML==6.0.1
readchar==4.0.5
referencing==0.30.2
regex==2023.10.3
requests==2.31.0
requests-oauthlib==1.3.1
rich==13.6.0
rpds-py==0.10.6
rsa==4.9
ruamel.yaml==0.17.39
ruamel.yaml.clib==0.2.8
s3fs==2023.10.0
sentry-sdk==1.32.0
six==1.16.0
sniffio==1.3.0
sortedcontainers==2.4.0
SQLAlchemy==2.0.22
starlette==0.27.0
tblib==2.0.0
text-unidecode==1.3
toml==0.10.2
toolz==0.12.0
tornado==6.3.3
typer==0.9.0
typing_extensions==4.8.0
tzdata==2023.3
tzlocal==5.1
urllib3==1.26.18
uvicorn==0.23.2
websocket-client==1.6.4
websockets==11.0.3
wrapt==1.15.0
yarl==1.9.2
zict==3.0.0
zipp==3.17.0

[chrisguidry]

Thank you! We'll dig into this

[chrisguidry]

Hi @radiatorius1024, thanks for the bug report, we believe we have corrected this in the latest prefect release (2.14.1): https://github.com/PrefectHQ/prefect/releases/tag/2.14.1

Can you let us know if that corrected things for you?

[radiatorius1024]

Hi @radiatorius1024, thanks for the bug report, we believe we have corrected this in the latest prefect release (2.14.1): https://github.com/PrefectHQ/prefect/releases/tag/2.14.1

Can you let us know if that corrected things for you?

Everything works great now, thanks.

[jloehn]

Hey @chrisguidry I have been stuck on this same error for a while now.

I am using a docker work-pool to run the flow (and subflow) that's stored locally. I was initially using Prefect Server and Connection blocks to maintain the Airbyte Connection information, but have since adjusted my script to match radiatorius', but I'm still getting the same error.

Could it be the work-pool?

Here are my specs:

• Ubuntu 22.04.03 LTS
• prefect==2.14.1
• prefect-airbyte==0.3.2
• pydantic==2.5.2
• docker==6.1.3
• docker-compose==1.29.2
• prefect-airbyte==0.3.2
• prefect-docker==0.4.1

[chrisguidry]

Sorry about that, @jloehn! Can you paste the full exception you're seeing with the stack trace so we can see where it's happening for you? It may be the same error but manifesting from a different place.

[jloehn]

Not sure why the stack trace is showing python 3.10 as library. I updated to 3.11 and removed all 3.10 libraries. Could that be causing it? Or would it be something else with the worker-pool?

[chrisguidry]

Hmm, if you've updated to python 3.11, is it possible this image itself is outdated? Are you able to easily modify this flow to print out some info from inside the container? Maybe something like:

logger = get_run_logger()

logger.info("prefect version %s", prefect.__version__)
logger.info("prefect_airbyte version %s", prefect_airbyte.__version__)
logger.info("pydantic version %s", pydantic.__version__)

from within the flow so we can see if maybe it's not running the latest versions?

[jloehn]

@chrisguidry Thanks for pointing me in the right direction. I kept overlooking the push section of my prefect.yaml.

[chrisguidry]

Nice! Glad to hear it's working!