node-vlive-downloader

An automatic Vlive video downloader which, Given the URI:

1. Automatically downloads the dynamically created JSON file.
2. Downloads the Video format specified by a user string.
3. Merges ALL the available subtitles with the file specified above into an *.mkv file

Roadmap

• Command line interface to run globally
• Write Proper tests

Testing

You can run the following which will attempt to gather metadata for a video, output the metadata, and download all subtitles and then attempt to download a 1080p video and then merge it in.

node test/advanced.js

Changelog

v0.3.0

Downloader works as of 01/09/2021

Its been ~7months since my last update, and I just want to say: "vLive is doing something."

Version bump because I needed to update the package dependencies due to security issues.

My tests failed for two reasons:

1. I updated my NodeJS environment so I had to change the way the tests use the imported library. I ended up back tracking on this because it was causing too much problems.
2. vLive changed something in their Likes CSS Selector which broke my entire application - My Fault really.

I proceeded to then update the library with the following:

• Added a new catch() to each chain that asks the web page for values. This ensures that the Application can continue running and does not hang. If you encounter a "X Cannot Be Parsed" anywhere in your main Metadata file it means that vLive has updated their Web page.
• Fixed an error which caused the browser to hang around after the web scraper failed for any reason.
• Began writing in Comments in the source code for me and whoever may need to follow along.

2021 Update - v0.2

It has been almost two years since my last update, but driven by the recent forks and the star of this repo, I decided to fix it. The library originally broke with a VLive update sometime in 2019/2020. I wasn't keeping track of it. However, it seems that they did another update which, with a few tweaks to the parser I was able to have the library work just as before. So thank you to the two individuals for making me look at this library again.

Also, some of the Readme needs to be updated as the structure of some of the meta data has changed, however, I wanted to quickly get this update out to the people who would like it.

• Library now works with new VLive Update as of 2021/02/07.
• Updated Browser parser to look for the JSON, and metadata, in the new locations.

Hotfix - v0.1.5

• Added a new proper working test, test/advanced.js. Disregard test/basic_test.js
• Ensures that the library doesn't fail/crash/error out when a video with no captions is selected to download.

Meta - v0.1.4

• Added the ability to collection additional metadata from the video's page.

Variable Name	typeof	What is it
plays	integer	Number of times the video has been played.
likes	integer	Number of times the video has been liked.
date	string	The approximated date the video was published. Due to the nature of VLive's time backdating we can not get an accurate timestamp, only an estimated DATE.

History

I originally wrote a CLI version of this using child_process and ffmpeg after seeing @drklee3 's version of a vlive-downloader. Their code was great, but required more than just ffmpeg and it required you to manually get the vod_play_videoInfo.json file and pass it in as a command function.

The other version I spoke about above required two separate programs installed in other to generate the *.mkv files. node-vlive-downloader only uses ffmpeg to do everything basically.

Prerequisites

1. Requires ffmpeg to be installed on the host machine.
2. [OPTIONAL] Requires an instance of Chrome. Sometimes the internal Chromium build may not be able to get the data, and Chrome might be needed.

Installation

Installing the Library

yarn add node-vlive-downloader

Usage

When using the library in your code, most of the libraries functions can either be used via Promises or using Async/Await.

Full Example

This is a general idea of how the library can be used. The API below this code block will explain in detail the features and return values.

var browser = (require("node-vlive-downloader")()).vBrowser
var helper = (require("node-vlive-downloader")()).Helper
const url = "YOUR URL HERE";


(async function() {
  // get meta data from browser
  let metadata = await browser.getMetaData(`${url}`)
  // load it into memory
  let memory = await helper.saveToMemory(metadata.vod_url)
  // parse byte data into JS Object
  let vodData = await helper.parseJson(memory)
  // just download all the subs
  let subtiles = await helper.downloadAllSubtitles(vodData.video_captions.list)
  // find the meta data of the video we want to download
  let video2dl = await helper.getVideoEncode('1080')
  // download the file, and keep track of its location
  let dl = await helper.downloadVideoFile(video2dl, function(progress) {
    console.log(progress)
  })
  // merge the file.
  await helper.MergeIntoMKV(dl, subtiles, video2dl)
})()

Getting Meta Data from URI

This will launch a headless Chromium browser, that will attach listeners to its network requests. Once it detects the dynamically created JSON file it will store its location, and once networkidle0 fires, it will scrape the page for the additional meta data.

Example code using Promises

let browser = (require('node-vlive-downloader')()).vBrowser
const url = "YOUR_URL_HERE"

browser.getMetaData(`${url}`).then(metadata => {
  console.log(metadata)
}).catch(error => {
  console.error(error)
})

Example code using Async/Await

let browser = (require('node-vlive-downloader')()).vBrowser
const url = "YOUR_URL_HERE";

(async function() {
  let metadata = await browser.getMetaData(`${url}`)
  console.log(metadata)
})()

vBrowser.getMetaData Returns

Variable Name	typeof	What is it
vod_url	string	A URI to the JSON File which contains the raw meta data that VLive's player uses.
channel	string	Name of the Channel that uploaded the video.
title	string	Name of the video being requested.
plays	integer	Number of times the video has been played.
likes	integer	Number of times the video has been liked.
date	string	The approximated date the video was published. Due to the nature of VLive's time backdating we can not get an accurate timestamp, only an estimated DATE.

Downloading The vod_play_videoInfo.json file

Once you have the link, you can either save it to disk, or you can save it to memory. You can manually process this file/link, or you can use the built-in model schemas to then filter the data you want.

Example code where you save it to disk

(async function() {
  let metadata = await browser.getMetaData(`${url}`)
  console.log(metadata)
  let savetodisk = await helper.saveToDisk(metadata.vod_url)
  console.log(savetodisk)
})()

Helper.saveToDisk Returns

Variable Name	typeof	What is it
returns	string	A path to saved file. Defaults to tmp/vod_play_videoInfo.json

Example code where you save it to memory

(async function() {
  let metadata = await browser.getMetaData(`${url}`)
  console.log(metadata)
  let memory = await helper.saveToMemory(metadata.vod_url)
  console.log(memory)
})()

Helper.saveToMemory Returns

Variable Name	typeof	What is it
returns	json	Once the buffer is downloaded via the url, it is parsed, and will return a json object if successful.

Parsing the JSON File

Once you have saved the file either to memory or to disk, you will need to reload it for node-vlive-downloader to be able to use it to process data. This parsing process loads the varying data from vod_play_videoInfo.json into a more structured schema, one that node-vlive-downloader understands.

Example parsing the JSON object from disk.

(async function() {
  // ...
  // load from browser into memory/disk
  let metadata = JSON.parse(fs.readFileSync('tmp/vod_play_videoInfo.json'))
  let vodData = await helper.parseJson(metadata)
  console.log(vodData)
})()

Helper.parseJson Returns

Variable Name	typeof	What is it
returns	json	Passing in a string, buffer, or JSON object, will yield a JSON object if successful, or an error if not.

Downloading Subtitles

You can download the subtitles either individually or you can download the entire batch. You can pass in a list of language locales, such as ko_KR, ja_JP, en_US etc. and the library will download those and convert them in to appropriate format.

NB At this point, the library expects that you have parsed the JSON file into the Helper object under require('node-vlive-downloader')().

Example of downloading a single subtitle.

(async function() {
  // ...
  // load from browser into memory/disk
  let metadata = JSON.parse(fs.readFileSync('tmp/vod_play_videoInfo.json'))
  let vodData = await helper.parseJson(metadata)
  let subtiles = await helper.downloadSubtitles('en', vodData.video_captions.list)
  console.log(subtiles)
})()

Helper.downloadSubtitles Returns

Variable Name	typeof	What is it
returns	array	Array of SubtitleMeta Objects. (See Schemas Sections)

Example of downloading all available.

(async function() {
  // ...
  // load from browser into memory/disk
  let metadata = JSON.parse(fs.readFileSync('tmp/vod_play_videoInfo.json'))
  let vodData = await helper.parseJson(metadata)
  let subtiles = await helper.downloadAllSubtitles(vodData.video_captions.list)
  console.log(subtiles)
})()

Helper.downloadAllSubtitles Returns

Variable Name	typeof	What is it
returns	array	Array of SubtitleMeta Objects. (See Schemas Sections)

Downloading Video Encodes

The library assumes that you will only want one video file. You can pick out the best quality one manually if you know the resolutions off hand. Or you can list the formats and then pick from there.

Example of listing all the video formats available.

(async function() {
  // ...
  // load from browser into memory/disk
  let metadata = JSON.parse(fs.readFileSync('tmp/vod_play_videoInfo.json'))
  let vodData = await helper.parseJson(metadata)
  let videoFormats = await helper.getVideoFormatEncodes(vodData.video_list)
  console.log(videoFormats)
})()

Helper.getVideoFormatEncodes Returns

Variable Name	typeof	What is it
returns	array	Array of VideoMeta Objects. (See Schemas Sections)

Example Of picking the video format you want.

(async function() {
  // ...
  // load from browser into memory/disk
  let metadata = JSON.parse(fs.readFileSync('tmp/vod_play_videoInfo.json'))
  let vodData = await helper.parseJson(metadata)
  let videoFormat = await helper.getVideoEncode('1080')
  console.log(videoFormat)
})()

Helper.getVideoEncode Returns

Variable Name	typeof	What is it
returns	VideoMeta	VideoObjectSchema object (See Schemas Sections)

Example of actually downloading the file you want.

(async function() {
  // ...
  // load from browser into memory/disk
  let metadata = JSON.parse(fs.readFileSync('tmp/vod_play_videoInfo.json'))
  let vodData = await helper.parseJson(metadata)
  let subtiles = await helper.getVideoEncode('1080')
  let dl = await helper.downloadVideoFile(subtiles, function(progress) {
    console.log(progress)
  })
  console.log(dl)
})()

Helper.downloadVideoFile Returns

Variable Name	typeof	What is it
returns	LocalVideoMeta	LocalVideoObjectSchema object (See Schemas Sections)

Merging Subtitles And Video File

We are basically done here. You can manually load up the video file which was stored under tmp/video and the subtitle you want, stored under tmp/srt. However, it would be far more convenient to merge them automagically into a container that supports both.

Function Expectations

The function expects certain objects to be passed into it to work properly.

Variable Name	Schema Name	What Returns It
downloadedFile (arg=0)	LocalVideoObjectSchema	Helper.downloadVideoFile
srt_subtitles (arg=1)	SubtitleMeta	Helper.downloadAllSubtitles Helper.downloadSubtitles
selectedFile (arg=3)	VideoObjectSchema	Helper.getVideoEncode

async MergeIntoMKV(downloadedFile = LocalVideoObjectSchema,
                   srt_subtitles = array(SubtitleMeta),
                   selectedFile = VideoObjectSchema)

Example of using the method.

(async function() {
  // ...
  // load from browser into memory/disk
  let metadata = JSON.parse(fs.readFileSync('tmp/vod_play_videoInfo.json'))
  let vodData = await helper.parseJson(metadata)
  let subtiles = await helper.downloadAllSubtitles(vodData.video_captions.list)
  let video2dl = await helper.getVideoEncode('1080')
  let dl = await helper.downloadVideoFile(video2dl, function(progress) {
    console.log(progress)
  })
  await helper.MergeIntoMKV(dl, subtiles, video2dl)
})()

Schemas

SubtitleMeta

{
  path,
  lang,
  locale,
  filename,
  format,
  country,
  label,
  type,
  fan
}

VideoObjectSchema

{
  id,
  duration,
  fileSize,
  videoType,
  encodingData = {
    encodeId
    name
    profile
    width
    height
    videoBitrate
    audioBitrate
  },
  source
}

LocalVideoObjectSchema

{
  path
}

License/Author Info

Project developed and maintained by VishalRamki. The project is licensed under the MIT license.