feat(lyrics-plus): add japanese lyrics conversion tool

[41pha1]

This adds the functionality of switching between different display modes for Japanese lyrics.

[theRealPadster]

Probably should have the dictionaries included in the same repo

[41pha1]

Probably should have the dictionaries included in the same repo

Alright I included them in the PR now

[kyrie25]

This is really amazing work! 😄 Thanks a lot for your contribution.
I just have a few small suggestions regarding this PR.

[CharlieS1103]

How about we make it so lyrics plus uses his translation or code or whatever initially with just hotloading but if the user installs the dicts locally via an install script then it uses that method instead? So users who don't need the feature don't need the extra 30mb and the users who often use the feature and don't want the load times can install it

[41pha1]

The dictionaries will now be dynamically loaded if they are not yet present and the conversion menu will show a loading indicator.

If the dicts are somehow added to the client in any other way, it will use them instead and skip the downloading step.

The download urls are in Translator.js line 6050
(I havent come up with a better place than my personal repo yet)

[rxri]

Could you provide source code for this then which I will upload to repo?

[41pha1]

Do you mean the code of the modified kuroshiro / analyzer bundle that currently sits along the Translator class in Translator.js?
I sadly don't have separated files anywhere as I exported everything directly to the bundle with npm.

I am not quite sure if I understand correctly :/

[rxri]

So, you don't have any source-code for this code? We will not be able to update a code then. We can't extract source from webpack compiled code.

[41pha1]

Oh yea my bad! I added the unminified code and split it into kuromoji.js and kuroshiro.js If wanted we can move them to the separate repo?

[rxri]

Revert this commit please. I didn't mean unminified code but source code of it. This is still bundled by webpack which is not a source code

[41pha1]

I am afraid I cant provide you with what you are looking for in that case

[kyrie25]

I am afraid I cant provide you with what you are looking for in that case

We are asking for the original code in which this was compiled to, if that clarifies anything for you.

Currently what you have committed here is the minified/compiled code from a Webpack build process, of which we cannot inspect, verify and maintain its functionality.

[41pha1]

We are asking for the original code in which this was compiled to, if that clarifies anything for you.

Currently what you have committed here is the minified/compiled code from a Webpack build process, of which we cannot inspect, verify and maintain its functionality.

I will have to get to work on the files from the Kuroshiro repo again, because I was as naive as to modify the compiled code from npm directly and thus am not and never haven been in the possession of the uncompiled code.

[huhridge]

Is there a possibility of bundling the translate function within Spicetify? So maybe it can be used in other extensions and custom apps, etc?

[kyrie25]

Is there a possibility of bundling the translate function within Spicetify? So maybe it can be used in other extensions and custom apps, etc?

The extension will be released as a standalone package so feel free to use that instead. We have no plans to bundle it into the CLI, mostly because it has a very small use case, scope of usage and is 10 times larger the CLI itself.

[huhridge]

Yeah, but the problem is importing modules in the extensions and making it work with marketplace, unless that was fixed?

[kyrie25]

Yeah, but the problem is importing modules in the extensions and making it work with marketplace, unless that was fixed?

That would be discussed on the Marketplace side of development. You can always choose to opt out on having users install via Marketplace and do it locally instead. We are also planning to make an option for developers to redirect users to their repository's Readme instead of installing by default on Marketplace (spicetify/marketplace#346)

Either way, we are not forcing users to increase their installation package size by tenfold for a feature that has probably around 1-2% of the userbase using it.

[Pithaya]

In case it helps, I've also been trying to integrate Kuroshiro in a spicetify extension without having to download the dictionary files locally, and got it to work without having to host the files in a repo. You can directly get the dictionary files from npm.
There is an issue with loading external urls however (I guess this is why the library's code had to be edited here to use downloaded dictionary files directly?), so I had to add a fix (found here) to get it working.

The repo is private for now so I can't link the file but here is the code I'm using (the extension is built with spicetify creator):

import Kuroshiro from 'kuroshiro';
import KuromojiAnalyzer from '../../node_modules/kuroshiro-analyzer-kuromoji/dist/kuroshiro-analyzer-kuromoji.min.js';

type ExtendedXMLHttpRequest = XMLHttpRequest & {
    patched_open: (method: string, url: string) => void;
};

const dictPath = 'https://cdn.jsdelivr.net/npm/kuromoji@0.1.2/dict';
const dictPathWithoutDoubleSlash =
    'https:/cdn.jsdelivr.net/npm/kuromoji@0.1.2/dict';
/**
 * Fix an issue with kuromoji when loading dict from external urls
 * See: https://github.com/takuyaa/kuromoji.js/issues/37
 * Adapted from: https://github.com/mobilusoss/textlint-browser-runner/pull/7
 */
export function applyKuromojiFix(): void {
    (XMLHttpRequest.prototype as ExtendedXMLHttpRequest).patched_open =
        XMLHttpRequest.prototype.open;
    XMLHttpRequest.prototype.open = function (method: string, url: string) {
        if (url.indexOf(dictPathWithoutDoubleSlash) === 0) {
            (this as ExtendedXMLHttpRequest).patched_open(
                method,
                url.replace(dictPathWithoutDoubleSlash, dictPath)
            );
        } else {
            (this as ExtendedXMLHttpRequest).patched_open(method, url);
        }
    };
}

export async function createKuroshiro(): Promise<Kuroshiro> {
    applyKuromojiFix();

    const kuroshiro: Kuroshiro = new Kuroshiro();
    const analyzer: KuromojiAnalyzer = new KuromojiAnalyzer({
        dictPath: dictPath,
    });

    await kuroshiro.init(analyzer);

    return kuroshiro;
}

[OverRevvv]

Thank you for your contribution and this fantastic extension. I'd like to know if we can convert Romanji further into English.

[ohitstom]

i really hope this isnt abandoned lol

[41pha1]

I think I got it to work by loading kuroshiro, kuromoji and the dictionary files via JSDeliver directly from npm using the XMLHttp fix similar to the one suggested by Pithaya.

If you approve these changes I will look into resolving the merge conflicts

I have also seen a lot request for supporting romanization of more languages like Korean or Russian

[rxri]

It's okay by me, @41pha1

[kyrie25]

LGTM

[Tetrax-10]

@41pha1

In line 114 should that be isJapanese or !isJapanese Because if its a japanese song then the menu should show up right?