swarm-video-migrator

Getting started

Prerequisites

1. docker
2. docker-compose
3. youtube-dl binary (http://rg3.github.io/youtube-dl/)
4. ffmpeg binary
5. ffprobe binary

Installation

1. Create an .env file (see included .env.dist) and fill in the variable for the environment you want (e.g. prod or dev)
2. Create a your_env.json config file in config/ and name it the same as your intended environment (see included dev.json)
3. Download the required binaries from their respective websites and put them in volumes/vendor
4. docker-compose run producer npm i
5. docker-compose run consumer npm i
6. docker-compose run consumer npm run build
7. docker-compose run producer npm run build
8. docker-compose up (-d) producer
9. docker-compose up (-d) consumer

Endpoints^[1]

(EXTERNAL)^[2] /api/post/order

Post orders here.

Payload example:

{
    "url" : "https://www.youtube.com/watch?v=9pCYyrlnRiY",
    "swarm_hash" : "your_swarm_hash",
    "quality" : "hd",
    "upload_to" : "beefree",
    "output" : "video",
    "album_id" : 1
}

Payload keys:

1. "url" is the piece of content to download
2. "swarm_hash" is the location to upload the content to (beefree profile hash in the case of beefree)
3. "quality" : hd or sd
4. "upload_to" : beefree, swarm, or s3*
5. "output" : audio or video (video includes audio)

• *Be sure to fill out the required keys in the app config.

BeeFree specific:

1. "album_id" : an album id for BeeFree to upload the videos to. If value is undefined/null, then new album is created.

(INTERAL)^[3] /api/order/pop - GET

Query this endpoint to get a work order. This will be popped from the redis queue and marked as pulled by a consumer.

(INTERAL) /api/order/:uuid - GET

Query this endpoint to get a specific order by the uuid provided

(INTERAL) /api/order - POST

Query this endpoint to post a new order

(INTERAL) /api/order/status - POST

Query this endpoint to post a order update

Payload example:

{
    "uuid" : "https://www.youtube.com/watch?v=9pCYyrlnRiY",
    "status" : "your_status",
    "result" : "job_results" //optional
}

[Annotations]

1. All endpoints are configured to require a successful BASIC authentication with username/password.
2. External in the sense that this is an advertised endpoint for the outside to request new orders.
3. Internal in the sense that these are endpoints for communication between consumers and the producer.

Flow:

TL;DR:

Set env & config file, install docker (+compose), send an order to /api/post/order. Producer takes the order, registers it in the SQLite DB, and pushes a work order to the Redis queue. Consumers pull from the Redis queue through the producer, and does work on the order (downloading, transcoding, uploading).

Credit

1. youtube-dl is used for the downloading and transcoding of the videos, see http://rg3.github.io/youtube-dl/
2. ffmpeg and ffprobe are used as well.