Skip to content

Place wheels in content hashed folders when uploading to dbfs instead of renaming them#251

Merged
mukulmurthy merged 6 commits intodatabricks:pipelines-clientfrom
arulajmani:support-wheels
Aug 7, 2019
Merged

Place wheels in content hashed folders when uploading to dbfs instead of renaming them#251
mukulmurthy merged 6 commits intodatabricks:pipelines-clientfrom
arulajmani:support-wheels

Conversation

@arulajmani
Copy link
Copy Markdown
Contributor

Change the way we compute the hashed path for wheels. Wheels can't be renamed, so they are put in a folder with the same name as their content hash instead of the rename logic for all other file types.

@codecov-io
Copy link
Copy Markdown

codecov-io commented Aug 2, 2019

Codecov Report

Merging #251 into pipelines-client will decrease coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@                 Coverage Diff                  @@
##           pipelines-client     #251      +/-   ##
====================================================
- Coverage             83.35%   83.34%   -0.02%     
====================================================
  Files                    33       33              
  Lines                  2127     2131       +4     
====================================================
+ Hits                   1773     1776       +3     
- Misses                  354      355       +1
Impacted Files Coverage Δ
databricks_cli/pipelines/api.py 87.35% <100%> (+0.6%) ⬆️
databricks_cli/dbfs/dbfs_path.py 88.46% <0%> (-1.93%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cd43a93...f1e9d73. Read the comment docs.

Copy link
Copy Markdown
Collaborator

@mukulmurthy mukulmurthy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a good way of testing what an input path (like whl rel/path.whl) hashes to?

Copy link
Copy Markdown
Collaborator

@mukulmurthy mukulmurthy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@arulajmani arulajmani changed the title Update dbfs path logic for Pipelines for consistency Support uploading local Python wheel files when deploying pipelines Aug 7, 2019
@arulajmani arulajmani changed the title Support uploading local Python wheel files when deploying pipelines Place wheels in content hashed folders when uploading to dbfs instead of renaming them Aug 7, 2019
@mukulmurthy mukulmurthy merged commit 85dd1e5 into databricks:pipelines-client Aug 7, 2019
mukulmurthy pushed a commit that referenced this pull request Dec 5, 2019
… of renaming them (#251)

Change the way we compute the hashed path for wheels. Wheels can't be renamed, so they are put in a folder with the same name as their content hash instead of the rename logic for all other file types.
null-sleep pushed a commit that referenced this pull request May 18, 2020
… of renaming them (#251)

Change the way we compute the hashed path for wheels. Wheels can't be renamed, so they are put in a folder with the same name as their content hash instead of the rename logic for all other file types.
mukulmurthy pushed a commit that referenced this pull request Jun 1, 2020
… of renaming them (#251)

Change the way we compute the hashed path for wheels. Wheels can't be renamed, so they are put in a folder with the same name as their content hash instead of the rename logic for all other file types.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants