Build a Wheel Distribution without altering the folder structure

[Sahith-8055]

I'm trying to build a distribution package based on Wheel. Below is the folder structure that I currently have -

.
└── dbtsqlworkloadtest/
    ├── dbt/
    │   ├── analyses/
    │   ├── macros/
    │   ├── models/
    │   ├── seeds/
    │   ├── snapshots/
    │   └── tests/
    ├── docs/
    │   └── index.md
    ├── catalog.yaml
    ├── dbt_project.yml
    ├── mkdocs.yml
    ├── profiles.yml
    ├── pyproject.toml
    ├── README.md
    ├── setup.cfg
    └── version.py

Currently, the situation is that when trying to build the wheel and extracting it locally to see the contents, I'm unable to preserve the directory structure mentioned above (meaning the files ending with *.yaml and *.yml are moved to a different directory and the rest in the root directory). Is there any way to ensure that I get a .whl file (or distribution package) with the same directory structure?

Currently, I get the below structure (after extracting the wheel) by executing the below command -

python -m build --wheel

.
└── dbtsqlworkloadtest/
    ├── .
    ├── .
    ├── .
    └── dist/
        ├── dbt_workload-0.2.1.dev0+gfbb0c19.d20231109-py3-none-any/
        │   ├── dbt/
        │   │   ├── analyses/
        │   │   ├── macros/
        │   │   ├── tests/
        │   │   ├── snapshots/
        │   │   ├── models/
        │   │   └── seeds/
        │   ├── dbt_workload-0.2.1.dev0+gfbb0c19.d20231109.data/
        │   │   └── data/
        │   │       ├── catalog.yaml
        │   │       ├── dbt_project.yml
        │   │       └── profiles.yml
        │   └── dbt_workload-0.2.1.dev0+gfbb0c19.d20231109.dist-info/
        │       ├── METADATA
        │       ├── RECORD
        │       ├── top_level.txt
        │       └── WHEEL
        └── dbt_workload-0.2.1.dev0+gfbb0c19.d20231109-py3-none-any.whl

Below is my setup.cfg -

[metadata]
name = dbt_workload
author = Sahith Y
author_email = sahithyerram@gmail.com
description = This package is used as an input to the Micro Service
maintainer = Sahith Y
license = MIT
classifiers =
    Programming Language :: Python :: 3
    Operating System :: OS Independent
    Topic :: Software Development :: Build Tools
    License :: OSI Approved :: MIT License

[options]
python_requires = >=3.7
setup_requires =
    setuptools_scm

[options.packages.find]
exclude =
    docs*
    mkdocs.yml
    README.md

[options.package_data]
dbt_workload = version.py

[options.data_files]
. = *.yaml, dbt_project.yml, profiles.yml

[tool.setuptools_scm]
write_to = version.py
fallback_version = 0.1.0
version_scheme = semver
local_scheme = no-local-version

Any ideas / suggestions on how to preserve the structure of the directory while creating the Wheel Distribution. Thanks in advance.

[jwodder]

If your goal is for the YAML files to be in the root of the wheel, that is a bad idea, as, when your wheel is installed, those files will be placed directly in the Python environment's site-packages/ directory, without any namespacing structure to separate the files from YAML files from other packages doing the same thing (and, if enough packages do the same thing, you'd eventually get conflicts where the same root-level YAML path exists in the wheels for two different projects, leading to either files being overwritten or outright errors on installation).

If you don't want your files in the *.data/ folder, the next best thing is to move them inside your Python package (here, dbt/) and access them at runtime using importlib.resources.

[Sahith-8055]

Thanks for the response. But my intention behind creating a wheel is different as I would not be installing the wheel through pip rather I would use the wheel in a use-case wherein I need to download the wheel on my local machine by using the publicly available APIs (I need to push the wheel onto Gitlab Package Registry).

Can you suggest ways to achieve this? Meaning, I need to move the files from *.data onto the root directory (Same level as dbt/) as I couldn't fully grasp the solution provided by you. Thanks.

[jwodder]

I do not see a situation in which having YAML files in the root of a wheel, on the same level as dbt/, is a reasonable thing to do, and I will not help you with it. Instead, I recommend you put the files inside dbt/.

[Sahith-8055]

Thanks for the proposed recommendation @jwodder. I had given it a thought and I agree with you to not include the YAML files on the same level as dbt/. Can you suggest ways now to place the files inside dbt/ instead of having the files under *.data/?

[jwodder]

Just move them in your repository to inside dbt/, and then see https://setuptools.pypa.io/en/latest/userguide/datafiles.html if you need help getting the files to be present in the wheel.