Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

video management updates #38

Merged
merged 14 commits into from
Dec 21, 2023
Merged

video management updates #38

merged 14 commits into from
Dec 21, 2023

Conversation

mradamcox
Copy link
Collaborator

This PR will merge a few updates to the repair av pipeline in an effort to better handle large video files, and should also address some issues like #32.

Also, though there isn't a ticket for it, the /admin page has become practically unusable because the videos seem to be set to autoplay and/or the preview gifs can be very heavy files, being not a single thumbnail but multiple frames from each video (i.e. arbitrarily long based on the length of the original video).

Ultimately this comes down to some changes to the ffmpeg commands.

  • the preview gif is now a single frame from the start of the video
  • the videos are optimized for web playback with -movflags faststart
  • the videos are now explicitly encoded with the H.264 video codec (-c:v libx264)
  • as before, the audio stream is encoded with whatever the input file already uses (-c:a copy)
    • I tried aac for the audio codec, but consistently got a Pthread error when running yarn repair-av locally

Additionally, a new environment variable can be set to limit the number of files that are processed in a single workflow run (REPAIR_AV_FILE_LIMIT). This should help address issues like #30 in the future.

One final note: We are now handling much larger files than when the site was originally designed. Because of how intertwined the 1) video upload 2) video repair/transcode 3) user survey data collection, and 4) admin approval processes are, I've made changes within the existing workflow, instead of picking it back apart into its constituent pieces and augmenting them individually.

Thus, for very large files >2gb, we'll try handling the admin upload process like so:

  1. use /admin-upload as normal, but upload a small dummy video (not the real video)
    -- the uploaded video will be named with the new story id
  2. locally, use native ffpmeg to transode the video, and also create the preview gif
    -- ffmpeg -i video.mp4 -c:v libx264 -c:a aac -aac_coder fast -movflags faststart output.mp4
    -- the extra aac flags seem to be helpful but copy can also be used.
  3. rename the file to match the already uploaded dummy video
  4. upload the file directly to s3 and replace the dummy video, and also upload the preview gif to the proper location
  5. add the id to scripts/skip-repairs.txt so that the github workflow will not touch it

Copy link

netlify bot commented Dec 21, 2023

Deploy Preview for atlas-histories ready!

Name Link
🔨 Latest commit d164865
🔍 Latest deploy log https://app.netlify.com/sites/atlas-histories/deploys/658470df4a097500086faa56
😎 Deploy Preview https://deploy-preview-38--atlas-histories.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@mradamcox mradamcox merged commit 682443d into main Dec 21, 2023
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant