[CT-2457] [Bug] dbt deps commands error our in databricks DBFS

[anjutiwari]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

dbt deps command fails to run in databricks DBFS.

• deps command internally creating a symlink while running dbt deps command. After more debugging, I found dbt internally creating a symlink when installing dbt deps.

• In current code the supports_symlinks is based on if python lib(os) has the symlink method available or not. https://github.com/dbt-labs/dbt-core/blob/main/core/dbt/clients/system.py#L161

• As DBT is running on 3.7+ Python we can get rid of supports_symlinks method. This will be always true in databricks DBFS because getattr(os, "symlink", None) will never return False with the python versions dbt supporting

• DBFS can not work with symlink so we should use copytree.

https://stackoverflow.com/questions/64314341/creating-symbolic-link-in-a-databricks-filestores-folder

Expected Behavior

• If symlink fails copy directory

Steps To Reproduce

1. Install dbt under the DBFS path
2. Run dbt deps command

Relevant log output

No response

Environment

- OS: NA (DBFS)
- Python: 3.8
- dbt: dbt-databricks==1.2.0

Which database adapter are you using with dbt?

other (mention it in "Additional Context")

Additional Context

No response

[jtcohen6]

@anjutiwari Thanks for opening! It's a relatively uncommon use case, but if the fix seems straightforward, I'm very happy to have you take a look :)

[anjutiwari]

Hello @jtcohen6,
Thanks for looking into it. I have pushed my changes, please have a look.