Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace AtomicParsley with @orsetto/meta-writer #582

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 1 addition & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,7 @@ RUN apk add --no-cache git g++ make cmake linux-headers
COPY --from=installer /freyr/node_modules /freyr/node_modules
RUN go install github.com/tj/node-prune@1159d4c \
&& node-prune --include '*.map' /freyr/node_modules \
&& node-prune /freyr/node_modules \
# todo! revert to upstream when https://github.com/wez/atomicparsley/pull/63 is merged and a release is cut
&& git clone --branch 20230114.175602.21bde60 --depth 1 https://github.com/miraclx/atomicparsley /atomicparsley \
&& cmake -S /atomicparsley -B /atomicparsley \
&& cmake --build /atomicparsley --config Release
&& node-prune /freyr/node_modules

FROM alpine:3.18.3 as base

Expand All @@ -25,7 +21,6 @@ RUN apk add --no-cache bash nodejs python3 \
&& find /usr/lib/python3* \
\( -type d -name __pycache__ -o -type f -name '*.whl' \) \
-exec rm -r {} \+
COPY --from=prep /atomicparsley/AtomicParsley /bin/AtomicParsley

COPY . /freyr
COPY --from=prep /freyr/node_modules /freyr/node_modules
Expand Down
22 changes: 0 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,27 +129,6 @@ Here's a list of the metadata that freyr can extract from each streaming service

</details>

<details>
<summary>AtomicParsley >= 20230114</summary>

First, download the latest release for your individual platforms here <https://github.com/miraclx/atomicparsley/releases/latest>

Then;

- Windows:
- unzip and place the `AtomicParsley.exe` in your `PATH`.
- or the `bins/windows` folder of this project directory. Create the folder(s) if they don't exist.
- Linux + macOS:
- unzip and place the `AtomicParsley` in your `PATH`.
- or the `bins/posix` folder of this project directory. Create the folder(s) if they don't exist.
- Alternatively:
- Debian: `sudo apt-get install atomicparsley`
- Arch Linux: `sudo pacman -S atomicparsley`
- Android (Termux): `apt install atomicparsley`
- Build from source: See [wez/AtomicParsley](https://github.com/miraclx/atomicparsley)

</details>

> _Please note that [YouTube Music](https://music.youtube.com/) must be available in your region for freyr to successfully work, this is because freyr sources raw audio from [YouTube Music](https://music.youtube.com/)._

---
Expand Down Expand Up @@ -305,7 +284,6 @@ Options:

Environment Variables:
SHOW_DEBUG_STACK show extended debug information
ATOMIC_PARSLEY_PATH custom AtomicParsley path, alternatively use `--atomic-parsley`

Info:
When downloading playlists, the tracks are downloaded individually into
Expand Down
176 changes: 57 additions & 119 deletions cli.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ import xurl from 'url';
import util from 'util';
import xpath from 'path';
import crypto from 'crypto';
import {spawn, spawnSync} from 'child_process';
import {promises as fs, constants as fs_constants, createReadStream, createWriteStream} from 'fs';

import Conf from 'conf';
Expand All @@ -28,6 +27,7 @@ import {program as commander} from 'commander';
import {decode as entityDecode} from 'html-entities';
import {createFFmpeg, fetchFile} from '@ffmpeg/ffmpeg';
import ProgressBar, {getPersistentStdout} from 'xprogress';
import metaWriter from '@orsetto/meta-writer';

import _merge from 'lodash.merge';
import _mergeWith from 'lodash.mergewith';
Expand Down Expand Up @@ -85,58 +85,7 @@ async function isOnline() {
}

function parseMeta(params) {
return Object.entries(params || {})
.filter(([, value]) => ![undefined, null].includes(value))
.map(([key, value]) =>
Array.isArray(value) ? value.map(tx => (tx ? [`--${key}`, ...(Array.isArray(tx) ? tx : [tx])] : '')) : [`--${key}`, value],
)
.flat(Infinity);
}

function extendPathOnEnv(path) {
return {
...process.env,
PATH: [path, process.env.PATH].join(process.platform === 'win32' ? ';' : ':'),
};
}

function ensureBinExtIfWindows(isWin, command) {
return command.replace(/(\.exe)?$/, isWin ? '.exe' : '$1');
}

function check_bin_is_existent(bin, path) {
const isWin = process.platform === 'win32';
const command = isWin ? 'where' : 'which';
const {status} = spawnSync(ensureBinExtIfWindows(isWin, command), [bin], {
env: extendPathOnEnv(path),
});
if ([127, null].includes(status)) throw Error(`Unable to locate the command [${command}] within your PATH`);
return status === 0;
}

function wrapCliInterface(binaryNames, binaryPath) {
binaryNames = Array.isArray(binaryNames) ? binaryNames : [binaryNames];
const isWin = process.platform === 'win32';
const path = xpath.join(__dirname, 'bins', isWin ? 'windows' : 'posix');

if (!binaryPath) {
for (let name of binaryNames) {
if (!check_bin_is_existent(name, path)) continue;
binaryPath = ensureBinExtIfWindows(isWin, name);
break;
}
if (!binaryPath)
throw new Error(
`Unable to find an executable named ${(a =>
[a.slice(0, -1).join(', '), ...a.slice(-1)].filter(e => e != '').join(' or '))(binaryNames)}. Please install.`,
);
} else binaryPath = xpath.resolve(binaryPath);
return (file, args, cb) => {
if (typeof file === 'string')
spawn(binaryPath, [file, ...parseMeta(args)], {
env: extendPathOnEnv(path),
}).on('close', cb);
};
return Object.fromEntries(Object.entries(params || {}).filter(([, value]) => !['', undefined, null].includes(value)));
}

function getRetryMessage({meta, ref, retryCount, maxRetries, bytesRead, totalBytes, lastErr}) {
Expand Down Expand Up @@ -758,22 +707,6 @@ async function init(packageJson, queries, options) {
),
);

let atomicParsley;

try {
let atomicParsleyPath = options.atomicParsley || process.env.ATOMIC_PARSLEY_PATH;
if (atomicParsleyPath) {
if (!(await maybeStat(atomicParsleyPath)))
throw new Error(`\x1b[31mAtomicParsley\x1b[0m: Binary not found [${options.atomicParsley}]`);
if (!(await isBinaryFile(atomicParsleyPath)))
stackLogger.warn('\x1b[33mAtomicParsley\x1b[0m: Detected non-binary file, trying anyways...');
}
atomicParsley = wrapCliInterface(['AtomicParsley', 'atomicparsley'], atomicParsleyPath);
} catch (err) {
stackLogger.error(err.message);
process.exit(7);
}

async function createPlaylist(header, stats, logger, filename, playlistTitle, shouldAppend) {
if (options.playlist !== false) {
const validStats = stats.filter(stat => (stat[symbols.errorCode] === 0 ? stat.complete : !stat[symbols.errorCode]));
Expand Down Expand Up @@ -1096,56 +1029,62 @@ async function init(packageJson, queries, options) {
Config.concurrency.embedder,
async ({track, meta, files, audioSource}) => {
try {
await Promise.promisify(atomicParsley)(meta.outFile.path, {
overWrite: '', // overwrite the file

title: track.name, // ©nam
artist: track.artists[0], // ©ART
composer: track.composers, // ©wrt
album: track.album, // ©alb
genre: (genre => (genre ? genre.concat(' ') : ''))((track.genres || [])[0]), // ©gen | gnre
tracknum: `${track.track_number}/${track.total_tracks}`, // trkn
disk: `${track.disc_number}/${track.disc_number}`, // disk
year: new Date(track.release_date).toISOString().split('T')[0], // ©day
compilation: track.compilation, // ©cpil
gapless: options.gapless, // pgap
rDNSatom: [
// ----
['Digital Media', 'name=MEDIA', 'domain=com.apple.iTunes'],
[track.isrc, 'name=ISRC', 'domain=com.apple.iTunes'],
[track.artists[0], 'name=ARTISTS', 'domain=com.apple.iTunes'],
[track.label, 'name=LABEL', 'domain=com.apple.iTunes'],
[`${meta.service[symbols.meta].DESC}: ${track.uri}`, 'name=SOURCE', 'domain=com.apple.iTunes'],
[
`${audioSource.service[symbols.meta].DESC}: ${audioSource.source.videoId}`,
'name=PROVIDER',
'domain=com.apple.iTunes',
await metaWriter(
parseMeta({
TrackTitle: track.name, // ©nam
TrackArtist: track.artists[0], // ©ART
Composer: track.composers, // ©wrt
AlbumTitle: track.album, // ©alb
Genre: (genre => (genre ? genre.concat(' ') : ''))((track.genres || [])[0]), // ©gen | gnre
TrackNumber: `${track.track_number}`, // trkn
TrackTotal: `${track.total_tracks}`,
DiscNumber: `${track.disc_number}`, // disk
DiscTotal: `${track.disc_number}`,
RecordingDate: new Date(track.release_date).toISOString().split('T')[0], // ©day

AlbumArtist: track.album_artist, // aART
CopyrightMessage: track.copyrights.sort(({type}) => (type === 'P' ? -1 : 1))[0]?.text, // cprt
EncoderSoftware: `freyr-js cli v${packageJson.version}`, // ©too
EncodedBy: 'd3vc0dr', // ©enc
FrontCover: files.image.file.path, // covr

// Ilst tags
cpil: track.compilation, // cpil
stik: 'Normal', // stik
pgap: options.gapless, // pgap
rDNS: [
// ----
{mean: 'com.apple.iTunes', name: 'MEDIA', data: 'Digital Media'},
{mean: 'com.apple.iTunes', name: 'ISRC', data: track.isrc},
{mean: 'com.apple.iTunes', name: 'ARTISTS', data: track.artists[0]},
{mean: 'com.apple.iTunes', name: 'LABEL', data: track.label},
{mean: 'com.apple.iTunes', name: 'SOURCE', data: `${meta.service[symbols.meta].DESC}: ${track.uri}`},
{
mean: 'com.apple.iTunes',
name: 'PROVIDER',
data: `${audioSource.service[symbols.meta].DESC}: ${audioSource.source.videoId}`,
},
],
],
advisory: ['explicit', 'clean'].includes(track.contentRating) // rtng
? track.contentRating
: track.contentRating === true
? 'explicit'
: 'Inoffensive',
stik: 'Normal', // stik
// geID: 0, // geID: genreID. See `AtomicParsley --genre-list`
// sfID: 0, // ~~~~: store front ID
// cnID: 0, // cnID: catalog ID
albumArtist: track.album_artist, // aART
// ownr? <owner>
purchaseDate: 'timestamp', // purd
apID: 'cli@freyr.git', // apID
copyright: track.copyrights.sort(({type}) => (type === 'P' ? -1 : 1))[0]?.text, // cprt
encodingTool: `freyr-js cli v${packageJson.version}`, // ©too
encodedBy: 'd3vc0dr', // ©enc
artwork: files.image.file.path, // covr
// sortOrder: [
// ['name', 'NAME'], // sonm
// ['album', 'NAME'], // soal
// ['artist', 'NAME'], // soar
// ['albumartist', 'NAME'], // soaa
// ],
});
ParentalAdvisory: ['explicit', 'clean'].includes(track.contentRating) // rtng
? track.contentRating
: track.contentRating === true
? 'explicit'
: 'Inoffensive',
// geID: 0, // geID: genreID. See `AtomicParsley --genre-list`
// sfID: 0, // ~~~~: store front ID
// cnID: 0, // cnID: catalog ID
// ownr? <owner>
purd: new Date().toISOString(), // purd
apID: 'cli@freyr.git', // apID
// sortOrder: [
// ['name', 'NAME'], // sonm
// ['album', 'NAME'], // soal
// ['artist', 'NAME'], // soar
// ['albumartist', 'NAME'], // soaa
// ],
}),
meta.outFile.path,
);
} catch (err) {
throw {err, [symbols.errorCode]: 8};
}
Expand Down Expand Up @@ -1886,7 +1825,6 @@ function prepCli(packageJson) {
console.log('');
console.log('Environment Variables:');
console.log(' SHOW_DEBUG_STACK show extended debug information');
console.log(' ATOMIC_PARSLEY_PATH custom AtomicParsley path, alternatively use `--atomic-parsley`');
console.log('');
console.log('Info:');
console.log(' When downloading playlists, the tracks are downloaded individually into');
Expand Down
Loading
Loading