Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Single download pipeline #203

Merged
merged 3 commits into from
Jul 3, 2023
Merged

Single download pipeline #203

merged 3 commits into from
Jul 3, 2023

Conversation

mjh1
Copy link
Contributor

@mjh1 mjh1 commented Jun 29, 2023

Initial step towards avoiding task runner needing to download the source file. We check if the file already exists (from catalyst) and skip making the copy at task runner if so. Both cases have been tested by using a local version of catalyst-api with the source copy happening and without.

Also removing the metadata file which we deemed is not needed.

@mjh1 mjh1 requested a review from a team as a code owner June 29, 2023 10:15
@codecov
Copy link

codecov bot commented Jun 29, 2023

Codecov Report

Merging #203 (001c609) into main (fda26ec) will decrease coverage by 0.01242%.
The diff coverage is 0.00000%.

Impacted file tree graph

@@                 Coverage Diff                 @@
##                main        #203         +/-   ##
===================================================
- Coverage   14.78992%   14.77750%   -0.01242%     
===================================================
  Files             19          19                 
  Lines           2380        2382          +2     
===================================================
  Hits             352         352                 
- Misses          2003        2005          +2     
  Partials          25          25                 
Impacted Files Coverage Δ
task/upload.go 19.26978% <0.00000%> (-0.07849%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fda26ec...001c609. Read the comment docs.

Impacted Files Coverage Δ
task/upload.go 19.26978% <0.00000%> (-0.07849%) ⬇️

@mjh1 mjh1 requested review from victorges and leszko June 29, 2023 10:18
Copy link
Contributor

@leszko leszko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added one comment. Other than that, I have 2 concerns (discussed in Discord):

  1. Do we need this PR if we plan to remove copying from task-runner at all?
  2. The output asset has changed. Isn't that an issue?

task/upload.go Show resolved Hide resolved
@mjh1 mjh1 requested a review from leszko June 29, 2023 14:05
Copy link
Contributor

@leszko leszko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

task/upload.go Show resolved Hide resolved
task/upload.go Show resolved Hide resolved
@mjh1 mjh1 merged commit 84726bf into main Jul 3, 2023
13 checks passed
@mjh1 mjh1 deleted the mh/single-download branch July 3, 2023 10:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants