Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

api/vod: Implement last mile for Catalyst pipeline #1439

Merged
merged 14 commits into from
Nov 4, 2022

Conversation

victorges
Copy link
Member

What does this pull request do? Explain your changes. (required)

This implements the last missing pieces to get the E2E flow from Catalyst.

These felt merely bureaucratic so I didn't worry too much about getting it
implemented before we got an e2e flow with task-runner+catalyst first.
Ended up taking a couple of development hours tho so here I am, with the
testing session happening in less than a day lol

Specific updates (required)

  • Create separate OS config for catalyst files
  • Handle task output properly
  • Create files field in asset to hold the output from task-runner
  • Return proper playback and download URL for catalyst assets

-

  • How did you test each of these updates (required)
    Upload some files through VOD pipeline.

Does this pull request close any open issues?
Catalyst VOD.

Checklist:

  • I have read the CONTRIBUTING document.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.

We will be uploading files to the old default bucket (which can become
only for direct uploads), while the final files processed by catalyst
will go to a different bucket.

To fix, send the public URL of the file for processing instead of the
uploaded object key. That's what's gonna reach catalyst anyway.
@victorges victorges requested a review from a team as a code owner November 4, 2022 02:32
@vercel
Copy link

vercel bot commented Nov 4, 2022

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated
livepeer-studio ✅ Ready (Inspect) Visit Preview Nov 4, 2022 at 4:31PM (UTC)

Oh catalyst why do you keep calling after dark
@codecov
Copy link

codecov bot commented Nov 4, 2022

Codecov Report

Merging #1439 (1e73fcd) into master (46fe8e0) will increase coverage by 0.71300%.
The diff coverage is 60.00000%.

Impacted file tree graph

@@                 Coverage Diff                 @@
##              master       #1439         +/-   ##
===================================================
+ Coverage   52.28700%   53.00000%   +0.71300%     
===================================================
  Files             68          68                 
  Lines           4460        4500         +40     
  Branches         832         846         +14     
===================================================
+ Hits            2332        2385         +53     
+ Misses          1832        1809         -23     
- Partials         296         306         +10     
Impacted Files Coverage Δ
packages/api/src/parse-cli.ts 31.70732% <ø> (ø)
packages/api/src/controllers/task.ts 21.69811% <50.00000%> (+9.07675%) ⬆️
packages/api/src/controllers/asset.ts 59.34426% <53.65854%> (+3.35834%) ⬆️
packages/api/src/task/scheduler.ts 50.00000% <70.00000%> (+6.77966%) ⬆️
packages/api/src/controllers/playback.ts 86.48649% <71.42857%> (-4.68998%) ⬇️
packages/api/src/controllers/helpers.ts 49.35065% <100.00000%> (+0.30606%) ⬆️
packages/api/src/store/table.ts 73.24841% <0.00000%> (+3.18472%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d5fe23...1e73fcd. Read the comment docs.

Impacted Files Coverage Δ
packages/api/src/parse-cli.ts 31.70732% <ø> (ø)
packages/api/src/controllers/task.ts 21.69811% <50.00000%> (+9.07675%) ⬆️
packages/api/src/controllers/asset.ts 59.34426% <53.65854%> (+3.35834%) ⬆️
packages/api/src/task/scheduler.ts 50.00000% <70.00000%> (+6.77966%) ⬆️
packages/api/src/controllers/playback.ts 86.48649% <71.42857%> (-4.68998%) ⬇️
packages/api/src/controllers/helpers.ts 49.35065% <100.00000%> (+0.30606%) ⬆️
packages/api/src/store/table.ts 73.24841% <0.00000%> (+3.18472%) ⬆️

@victorges victorges merged commit 919536f into master Nov 4, 2022
@victorges victorges deleted the vg/fix/catalyst-vod-improvements branch November 4, 2022 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant