Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dagger excludes does not work for hidden files/folders #34070

Closed
bnchrch opened this issue Jan 9, 2024 · 2 comments · Fixed by #34136
Closed

Dagger excludes does not work for hidden files/folders #34070

bnchrch opened this issue Jan 9, 2024 · 2 comments · Fixed by #34136
Assignees

Comments

@bnchrch
Copy link
Contributor

bnchrch commented Jan 9, 2024

Steps to reproduce

  1. Add a .venv folder to any subdirectory that we load into Dagger
  2. In the pipeline that loads that directory log the context of that .venv directory
  3. Run the pipeline

Expected
Error as .venv folder should not exist

Actual
The .venv folder is in the dagger container

Why

The standard glob library in python 3.10 and lower does not follow hidden paths.
python/cpython#81759
https://stackoverflow.com/questions/49047402/python-glob-include-hidden-files-and-folders

Solutions

  1. Use something other than glob
  2. Update to 3.11
  3. Write a hack around
@alafanechere
Copy link
Contributor

Is it a Dagger problem or a problem in the client code (in the get_repo_dir https://github.com/airbytehq/airbyte/blob/main/airbyte-ci/connectors/pipelines/pipelines/models/contexts/pipeline_context.py#L217)

@bnchrch bnchrch self-assigned this Jan 11, 2024
@bnchrch
Copy link
Contributor Author

bnchrch commented Jan 11, 2024

Its a problem in our client code! Namely globs evaluating before working directory set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants