Airflow UI showing "No module named 'snowflake'" error after importing snowflake hook

[zeitgeistf]

Do not use issues for support requests or general help queries

Please ask on one of our mailing lists, or in our Slack channel. See the "Ask a question" section of http://airflow.apache.org/community/

Issues which are not bug reports or feature requests (i.e. "how do I get X working", or "How do I best do X?") will likely be closed without a response.

Hi all,

Recently my company plan to integrate our current data infrastructure with Snowflake, and we started setting up Snowflake hook in our existing Airflow project.

Unfortunately, with our current project environment setup, we have been seeing No module named 'snowflake' error as soon as we import SnowflakeHook from airflow.contrib.hooks.snowflake_hook import SnowflakeHook. (https://airflow.apache.org/docs/1.10.5/_modules/airflow/contrib/hooks/snowflake_hook.html#SnowflakeHook)

By looking at the Snowflake source code, we found that the first line in the hook file is trying to import snowflake.connector, and matches with the error message. In short, the snowflake hook failed to reference snowflake module upon importing.

Although we have found out where we are getting the error from, after hours of debugging and testing, we couldn't figure out how to resolve it. I am going to post our project env set up, and see if anyone else has encountered similar issues.

Airflow version: 1.10.5
Airflow root folder inside docker: root/airflow
Airflow directory tried to import hook: /airflow/dags/base_dag/flow

Pipfile:

[lucasveljacic]

I'm having the same error. Could you fix it?

[EvaGachirwa]

Hey @lucasveljacic, did you get a solution to this?

[potiuk]

We are just voting on new providers and snowflake providers are part of it (release candidates):

• Backport providers for 1.10: https://pypi.org/project/apache-airflow-backport-providers-snowflake/2021.2.5rc1/
• Providers for 2.0.0: https://pypi.org/project/apache-airflow-providers-snowflake/1.1.0rc1/

They are both bound with the new version of snowflake-connector-python > 2.3.8 which fixed some really annoying issues where snowflake broke other providers.

Maybe you can try this - if you are still on 1.10 install the backport provider, if you are on 2.0 install the regular one and try with the "airflow.providers.snowflake" operators.