Core dump using dbt-fal 1.5.4 with dbt-core 1.5.2

[kinghuang]

Describe the bug

I'm encountering a core dump when dbt-fal 1.5.4 is loaded with dbt-core 1.5.2.

dbt-fal 1.5.5 through 1.5.7 has a stated dependency for dbt-core >= 1.5, <= 1.5.1.

https://github.com/fal-ai/fal/blob/76d9d7502e4e16c0a02ca7387701a0b1cdb2e166/projects/adapter/pyproject.toml#L25

But, dbt-fal 1.5.4 and some earlier versions had a more permissive >=1.5.0, <1.6.

https://github.com/fal-ai/fal/blob/529c9311dcc90e1c46386b349e0629fe5fdef894/projects/adapter/pyproject.toml#L20

When I'm installing everything along with dbt-core==1.5.2 or dbt-core==1.5.3, this results in dbt-fal being resolved to 1.5.4. Is this a valid configuration? What is the reason for the tighter dependency in dbt-fal 1.5.5 through 1.5.7?

Your environment

• OS: macOS Ventura 13.4
• Paste the following commands output:

fal --version
bash: fal: command not found

dbt --version
Core:
  - installed: 1.5.3
  - latest:    1.5.3 - Up to date!

Plugins:
  - fal:              1.5.4 - Update available!
  - postgres:         1.5.3 - Up to date!
  - redshift:         1.5.8 - Up to date!
  - fal_experimental: 1.5.4 - Could not determine latest version

  At least one plugin is out of date or incompatible with dbt-core.
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

• Adapter being used: postgres

How to reproduce

pip install dbt-core dbt-fal[postgres]

Expected behavior

With a profile and project that uses fal, be able to run dbt debug.

Actual behavior

A core dump occurs when dbt loads fal. This can be reproduced to the following snippet.

from dbt.adapters.factory import load_plugin

load_plugin("fal")

Illegal instruction (core dumped)

Screenshots

root@f3aac2a15a3c:/debug# dbt debug
21:54:03  Running with dbt=1.5.2
21:54:03  dbt version: 1.5.2
21:54:03  python version: 3.11.4
21:54:03  python path: /usr/local/bin/python
21:54:03  os info: Linux-4.14.318-241.531.amzn2.aarch64-aarch64-with-glibc2.36
21:54:03  Using profiles.yml file at /debug/profiles.yml
21:54:03  Using dbt_project.yml file at /debug/dbt_project.yml
21:54:03  Configuration:
Illegal instruction (core dumped)

Additional context

[chamini2]

Hey @kinghuang , this is unexpected to me since our more strict dbt-core version is mostly for the dbt-fal CLI, not for the adapter. Curious if this happens for you for dbt-core==1.5.1 and latest dbt-fal version?

We had this working fine for 1.5.1 just recently.

[kinghuang]

I'm digging into this some more. It looks like I might be encountering two different problems, one for the core dump and one for the version pinning. I'll update this issue once I figure out the details.

[kinghuang]

I've tracked the core dump to NumPy, specifically when running in a container on Amazon Graviton instances. The issue is covered by numpy/numpy#24028.

dbt-fal indirectly imports numpy, hence why I was experiencing the problem when using a fal profile.

I'll close this issue, and open a new one about the detail in dbt-core dependencies. Thanks!

[chamini2]

Thank you for checking!