[CT-524] [Bug] Issue co-locating models and seeds in the same directory

[azhard]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

I’m trying to set the seed-paths to my models folder exactly like described in the docs but getting an error:

  dbt found two resources with the name "raw_preprocessor_internal_accounts". Since these resources have the same name,
  dbt will be unable to find the correct resource when source("raw_preprocessor", "internal_accounts") is used. To fix this,
  change the name of one of these resources:
  - source.dbt.raw_preprocessor.internal_accounts (models/raw_preprocessor/internal_accounts.yml)
  - source.dbt.raw_preprocessor.internal_accounts (models/raw_preprocessor/internal_accounts.yml)

Seems like this should be relatively straightforward, not sure why it’s identifying the same source as a duplicate of itself. Relevant parts of the dbt_project.yml file:

name: dbt

config-version: 2

version: 1.0.0

require-dbt-version: '>=1.0.0,<1.1.0'

profile: dbt

docs-paths:
  - models
macro-paths:
  - macros
model-paths:
  - models
seed-paths:
  - models
test-paths:
  - tests

log-path: logs
packages-install-path: dbt_packages
target-path: target

clean-targets:
  - target
  - dbt_packages

Expected Behavior

dbt is able to run (and compile) normally as it did before changing the seed-paths.

Steps To Reproduce

No response

Relevant log output

20:46:30  Running with dbt=1.0.5
20:46:30  Partial parse save file not found. Starting full parse.
20:46:33  No external sources selected
20:46:37  Encountered an error:
Compilation Error
  dbt found two resources with the name "raw_preprocessor_internal_accounts". Since these resources have the same name,
  dbt will be unable to find the correct resource when source("raw_preprocessor", "internal_accounts") is used. To fix this,
  change the name of one of these resources:
  - source.synctera_dbt.raw_preprocessor.internal_accounts (models/raw_preprocessor/internal_accounts.yml)
  - source.synctera_dbt.raw_preprocessor.internal_accounts (models/raw_preprocessor/internal_accounts.yml)

### Environment

```markdown
- OS: macOS 12.1
- Python: 3.1.
- dbt: 1.0.5

What database are you using dbt with?

BigQuery

Additional Context

No response

[emmyoop]

@azhard thanks for taking the time to submit a ticket and give some great details on reproducibility! I was able to reproduce this behavior. I have a few questions:

1. Are you upgrading from a previous version, are you rearranging your seeds in a current project to share the model path or are you starting a shiny new project?
2. Does your models directory contain anything other than .sql and .yml files?

While I was able to reproduce this, I am still working on tracking down the reason for this unexpected behavior.

I did discover a workaround that would at least allow you to move forward while I track this down. If you delete all the .yml files in /models, run dbt parse on your project (with success) and then re-add all the .yml file back exactly as they were, you should be able to move forward with development. I wanted to share that in case you are stuck at the moment while we work at figuring out the reason behind your issue.

[azhard]

Hey @emmyoop thanks for the response, to answer your questions:

1. Not upgrading, just finally getting around to some "tech-debt" of moving everything into the same folder and this is all from an existing project that started using dbt around version 0.20.0.
2. There were a few lingering .DS_STORE files but aside from that only .sql and .yml. Deleted all the .DS_STORE files to test but still had the same issue.

Gave the workaround a try but looks like getting a success will require a lot of manual tweaks cause of all the source dependencies for the model .sql files.

[gshank]

For some file categories we use an "all_source_paths" method which collects a number of directories from seed-paths, model-paths, snapshot-paths, analysis-paths and macro-paths. When two of them are the same the parsing code reads (and attempts to parse) them twice.

[azhard]

For some file categories we use an "all_source_paths" method which collects a number of directories from seed-paths, model-paths, snapshot-paths, analysis-paths and macro-paths. When two of them are the same the parsing code reads (and attempts to parse) them twice.

Would the fix here be something as simple as making this line return a list of unique elements?

[gshank]

Yes. That's what the pull request does :) #5176

[azhard]

Amazing - thanks for the quick turnaround!