add import newpipe subscriptions to plugin feed #72

aiosk · 2022-07-18T11:27:30Z

Condition:

i have more than 50 channel subscriptions on youtube
"Export to RSS readers" on https://www.youtube.com/subscription_manager is not working
i have move these channel subscriptions to newpipe subscriptions file (newpipe-subcriptions.json)

Expected:

i want to follow these youtube subscriptions via the plugin feed

Actual:

it is hardwork to add youtube channel subscriptions to plugin feed one by one. especially when you have more than 20
there is no way to import newpipe-subcriptions.json via plugin settings

Known Issue:
i have to save subscription path to settings.xml by click ok first before i can run import to feed.
ideally i want import feed after i set subscription path without click ok first

Reference:

Tips
convert youtube channel to newpipe-subcriptions.json

open https://www.youtube.com/feed/channels
login to your account
open web console. see firefox, chromium

copy paste this snippet, then press enter

snippet.js

ytContents = ytInitialData.contents.twoColumnBrowseResultsRenderer.tabs[0].tabRenderer.content.sectionListRenderer.contents;
window.prompt(
    'CTRL+A, CTRL+C',
    JSON.stringify({
        "app_version": "0.19.8", "app_version_int": 953,
        'subscriptions': ytContents[0].itemSectionRenderer.contents[0].shelfRenderer.content.expandedShelfContentsRenderer.items.map(item => ({
            'service_id': 0,
            'url': `https://www.youtube.com/channel/${item.channelRenderer.channelId}`,
            'name': item.channelRenderer.title.simpleText
        }))
    })
)

copy and paste dialog content to new file named newpipe-subcriptions.json

Condition: - i have multiple channel subscriptions on youtube - i have move these channel subscriptions to newpipe subscriptions file (newpipe-subcriptions.json) Expected: - i want to follow these youtube subscriptions via the plugin feed Result: - it is hardwork to add youtube channel subscriptions to plugin feed one by one - there is no way to import newpipe-subcriptions.json via plugin settings this patch add import newpipe-subcriptions.json to plugin feed. Reference: lekma#53

lekma · 2022-07-21T09:39:43Z

unfortunately I have no way to test all that at the moment, so I'll have to trust that you did test somewhat extensively, right?
i'd especially try to put weird unicode characters (emojis, and such) in you json file to test that they are imported correctly.
but if you can confirm that it's all alright, then, yeah, i would gladly merge.

for thre removeAll in persistence.py I would rename it to clear and call super on it (see SearchCache and SearchHistory in the same file).

thanks for the contribution

aiosk · 2022-07-21T13:07:31Z

unfortunately I have no way to test all that at the moment, so I'll have to trust that you did test somewhat extensively, right? i'd especially try to put weird unicode characters (emojis, and such) in you json file to test that they are imported correctly. but if you can confirm that it's all alright, then, yeah, i would gladly merge.

sorry, i only test it against my newpipe-subscriptions.json. I'll try my best to mitigate such case.

for thre removeAll in persistence.py I would rename it to clear and call super on it (see SearchCache and SearchHistory in the same file).

i will

thanks for your reply and suggestion

aiosk · 2022-07-21T16:27:24Z

@lekma,
i have no problem importing CJK characters and emoji in channel name, instead i have problem viewing these characters.
Here is my finding:

CJK characters

Condition

I have korean youtube channel with korean characters = https://www.youtube.com/channel/UCCsosBSGwE7TeaogXcmgV_Q
Kodi Settings > Interface > Skin > Skin = Estuary
Kodi Settings > Interface > Skin > Skin > Fonts = Skin default

Expected

kodi display korean characters on video title in Invidious > Feed
kodi display korean characters on Invidious > Settings > Persistent data > Remove channels feed from

Actual

kodi display empty box characters on video title in Invidious > Feed
kodi display empty box characters on Invidious > Settings > Persistent data > Remove channels feed from

Solution
i have to choose Arial based on Settings > Interface > Skin > Skin > Fonts to get kodi to display korean character

Emoji Characters

Condition

I edit newpipe-subscriptions.json so channel name contain emoji character
Settings > Interface > Skin > Skin = Estuary
Settings > Interface > Skin > Skin > Fonts = Arial based

Expected

kodi display emoji characters on video title in Invidious > Feed
kodi display emoji characters on Invidious > Settings > Persistent data > Remove channels feed from

Actual

kodi display empty box characters on video title in Invidious > Feed
kodi display empty box characters on Invidious > Settings > Persistent data > Remove channels feed from

Solution
i have to replace /usr/share/kodi/media/Fonts/arial.ttf with font GoNotoCJKCore.ttf from github.com/satbyy/go-noto-universal to get kodi to display emoji characters.
Even then, smilies emoji didn't work

This means:

You have to create new skin with font from github.com/satbyy/go-noto-universal set as Skin default, or
You have to add new font to kodi currently used skin, see how

lekma · 2022-07-22T06:48:30Z

@lekma, i have no problem importing CJK characters and emoji in channel name

good, this is what i wanted to confirm

instead i have problem viewing these characters.

this is an old kodi constraint/bug (it does not use system fonts), as you experienced, there isn't much we can do, as users, but tinker with the fonts...

question: is the clear() before importing really necessary? if you have common ids between the ones already existing and the ones being imported they will be updated, right? (there is no need to fix in the pr, i'll adjust before release)

request: would you mind attaching your json file (unless there is sensitive info inside) or point me to a sample i can download so that i can at least pretend i tried to test the feature 😄 ?

lekma · 2022-07-22T09:23:58Z

also, thanks again for the contribution

@aiosk: it seems I forgot to tag you

lekma · 2022-07-22T12:51:45Z

would you mind testing current master (or the file attached) and report?

thanks for your help
plugin.video.invidious-1.0.9-c6c9e76.zip

@aiosk: forgot to tag you

aiosk · 2022-07-22T16:50:22Z

question: is the clear() before importing really necessary? if you have common ids between the ones already existing and the ones being imported they will be updated, right? (there is no need to fix in the pr, i'll adjust before release)

no, .clear() is really not necessary.
I add .clear() because i really want the easy way to remove multiple subscriptions. As an alternative you can always remove channel from feed too..
come to think of it,, i think what i really want is more of sync subscriptions feature rather than just import.

request: would you mind attaching your json file (unless there is sensitive info inside) or point me to a sample i can download so that i can at least pretend i tried to test the feature smile ?

well i don't mind posting a subset of my json file..

newpipe-subscriptions.json

{
    "app_version": "0.19.8",
    "app_version_int": 953,
    "subscriptions": [
        {
            "service_id": 0,
            "url": "https://www.youtube.com/channel/UCGwyhqmF7HopVsbIigVTKmw",
            "name": "HIVI ©"
        },
        {
            "service_id": 0,
            "url": "https://www.youtube.com/channel/UCDom90xOqP4avehFjSJO6NA",
            "name": "Sick Science!"
        },
        {
            "service_id": 0,
            "url": "https://www.youtube.com/channel/UCH6vXjt-BA7QHl0KnfL-7RQ",
            "name": "Simon's Cat"
        },
        {
            "service_id": 0,
            "url": "https://www.youtube.com/channel/UCCsosBSGwE7TeaogXcmgV_Q",
            "name": "피클"
        }
    ]
}

aiosk · 2022-07-22T17:02:58Z

would you mind testing current master (or the file attached) and report? plugin.video.invidious-1.0.9-c6c9e76.zip

i tested it..
very nice..
exactly what i want..

thank you @lekma
and you're welcome

aiosk added 2 commits July 18, 2022 18:26

add new line at the end of strings.po

c9ff0bc

aiosk added 2 commits July 21, 2022 20:17

move ChannelFeed.removeAll() to ChannelFeed.clear()

a832b12

fixup! move ChannelFeed.removeAll() to ChannelFeed.clear()

cb904b7

lekma merged commit 6194141 into lekma:master Jul 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add import newpipe subscriptions to plugin feed #72

add import newpipe subscriptions to plugin feed #72

aiosk commented Jul 18, 2022 •

edited

lekma commented Jul 21, 2022

aiosk commented Jul 21, 2022 •

edited

aiosk commented Jul 21, 2022 •

edited

lekma commented Jul 22, 2022

lekma commented Jul 22, 2022 •

edited

lekma commented Jul 22, 2022 •

edited

aiosk commented Jul 22, 2022

aiosk commented Jul 22, 2022 •

edited

add import newpipe subscriptions to plugin feed #72

add import newpipe subscriptions to plugin feed #72

Conversation

aiosk commented Jul 18, 2022 • edited

lekma commented Jul 21, 2022

aiosk commented Jul 21, 2022 • edited

aiosk commented Jul 21, 2022 • edited

lekma commented Jul 22, 2022

lekma commented Jul 22, 2022 • edited

lekma commented Jul 22, 2022 • edited

aiosk commented Jul 22, 2022

aiosk commented Jul 22, 2022 • edited

aiosk commented Jul 18, 2022 •

edited

aiosk commented Jul 21, 2022 •

edited

aiosk commented Jul 21, 2022 •

edited

lekma commented Jul 22, 2022 •

edited

lekma commented Jul 22, 2022 •

edited

aiosk commented Jul 22, 2022 •

edited