Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nakamori as video library source #306

Closed
bigretromike opened this issue Apr 5, 2019 · 57 comments
Closed

Nakamori as video library source #306

bigretromike opened this issue Apr 5, 2019 · 57 comments
Labels
Kodi 18.x (Leia) It involves Kodi 18 compatibility Kodi 19.x (Matrix) It involves Kodi 19 compatibility New Feature A feature that will be part of Nakamori, but doesn't need new features from Shoko. Notes Notes for devs to keep ideas straight and/or remembered
Milestone

Comments

@bigretromike
Copy link
Owner

bigretromike commented Apr 5, 2019

This is very possible, this is something quite new, and no-one did it before.

xbmc/xbmc#13566
xbmc/xbmc#14210

I did somehow be able to make it work once, and then after cleaning library it broken - from last 8h I spend on this it looks like Library is buggy (and when cached something and remove it it still hold something for it, that would explain why it was working all the time until I wipe Library.db), like when you add something and it does scan, after many attempts and edits its start to throw errors like the path is not there. And I was hoping to make it without initially announcing it - but looks like I'm just a human.

When It did worked and added series it worked very well - with exception of missing few infos.

I came to conclusion that with all those custom object its utterly hared to do anything like this for me - and without proper IDE most of the time.

Currently its skips all the Series, the only thing is does scan are those extra menus if you add them to tvshow path. Which is strange.

I can be probably something trivial that I don't see, so fell free if you have spare time @da3dsoul to mess with this, as for now its not working.

I understand the cons of using nakamori as video source but I also see the potential and benefits for supporting this - this is far superior than any scraper for Shoko could be.

easy and dirty instruction is to add video source as plugin://plugin.video.nakamori/tvshows/<apikey>/, pick TV shows and Local information only in Set Content, double check if Selected folder contains a single TV show is uncheck and hit ok.

@bigretromike bigretromike added Notes Notes for devs to keep ideas straight and/or remembered Kodi 18.x (Leia) It involves Kodi 18 compatibility New Feature A feature that will be part of Nakamori, but doesn't need new features from Shoko. labels Apr 5, 2019
@bigretromike bigretromike added this to the long_term_goals milestone Apr 5, 2019
@bigretromike
Copy link
Owner Author

I would be glad if @AkariDN had a time to look it up, as he is the author of those awesome commits about this.

@da3dsoul
Copy link
Contributor

da3dsoul commented Apr 5, 2019

Well, following general Kodi and development guidelines. Please attach a debug log of Kodi when trying to do this.

Can you make a new repo plugin.videosource.nakamori? I don't know if there's a convention yet, but that sounds decent. We can move it into a new plugin so it's easy to enable and disable, as well as having its code unique to it, rather than plugin, which provides menus.

@bigretromike
Copy link
Owner Author

bigretromike commented Apr 5, 2019 via email

@da3dsoul
Copy link
Contributor

da3dsoul commented Apr 5, 2019

Then how do you turn it off?

@da3dsoul
Copy link
Contributor

da3dsoul commented Apr 5, 2019

Not everyone wants 45000 anime episodes in their collection of American Cartoons from the early 2000s

@bigretromike
Copy link
Owner Author

bigretromike commented Apr 5, 2019 via email

@bigretromike
Copy link
Owner Author

bigretromike commented Apr 5, 2019 via email

@da3dsoul
Copy link
Contributor

da3dsoul commented Apr 5, 2019

Ah I see.

@da3dsoul
Copy link
Contributor

da3dsoul commented Apr 5, 2019

oh....I need a nightly haha

@da3dsoul
Copy link
Contributor

da3dsoul commented Apr 5, 2019

I see one issue right off the bat. I'll see if I can get it working nicely.

@da3dsoul
Copy link
Contributor

da3dsoul commented Apr 5, 2019

@bigretromike don't feel bad. You found a bug...

@AkariDN The way to reproduce this issue (on latest nightly, 4/4/2019) is to make a URL pattern, for example plugin://plugin.video.nakamori/tvshows/12/ep/4, change it to something else, ex plugin://plugin.video.nakamori/tvshows/series/12/ep/4. When you enter the plugin, even from the Files menu, then it goes to the correct new URL. If you try to scan it, then it uses the old one, no matter what the code says. I'm looking into the db3 files next to see if I can find it.

EDIT: Verified. MyVideos116.db -> tables -> path is not updated, and it is used before trying the new URL.

@da3dsoul
Copy link
Contributor

da3dsoul commented Apr 5, 2019

kodi.log
Here's a debug log. It doesn't say anything helpful, though...

@da3dsoul
Copy link
Contributor

da3dsoul commented Apr 5, 2019

I'm having trouble getting anything to show up. If I enter the menus, then they all appear as expected, but the scan returns nothing.

@bigretromike
Copy link
Owner Author

I see you manage to find more than I did. The fun part is that if you type source as plugin://plugin.video.nakamori/ and you scan it you can see in top right cornet that it iterate over Search, Continue watching, etc (and see that search box apears, info box etc) its funny because I used filter_id=1 which gave me Continue watching and when it worked it did iterate over series and in right top corner there was full names of those series and later they would be added.

Later I was thinking that maybe /tvshow/ path in addon.xml is limiting it because it would enter tshows list items as url=/menu/series/<id> but from time to time when I change the code like chaning DirectoryList or ListItem to native xbmc object, or when I added other menus there I would see that it does enter into those episodes and most of the time hit /episode/types.

Funny how it work that one time because it was broken

The only thing you need to remember (but it looks like you know more than I did in 3 days) that when you use it a source for library it wont and CAN'T access settings - thats why some logic dont work.

The original idea was that I would anchor it on /tvshows/ that would list all items from All and it would flat the types and show only episodes and special. Funny enough when I did work it worked quite well it automaticly made Seasons for JoJo, proper naming etc....

But with the clue you found out about Library bugged not using url that would make sense because when I added it first I did change url many times, also I don't know if it should work on scaning for new content (from context menu) or it should be only tested from removing item, cleaning library, source.xml and then reading because other way it remembers settings etc.

I knew you will dig more in few hours than I did in few days, and it frustrating that we are 'thiiiis' close to solve it.

Hope its not another bug like the plugin_routing had.... ^_^

@bigretromike
Copy link
Owner Author

Before I would add tones of xbmc.log() in every path that I would think it goes and see in log where did he scan those files. Thats a good indicator what those it see.

And It does see each extra_menu from add_extra_menu (calendar, search, experiment) but refuse to see series which are kodi_models.ListItems inside DirectoryList. Thats why I wanted dump those, to compare because that could be something with that.

The last this is that I did try once (before it worked) with li.setProperty('get_stream_details_from_player', 'true') dunno if that helped or not, because I removed it later - but maybe it did (maybe its a clue).

But with the way Library is bugged the best way to test would be each time wipe clean kodi library and settings.... which would take ages.. or use container hahaha...

I'm just wishing that somehow we (You) manage to overcome this bug, because that would make a Nakamori even more useful.

Oh, the last thing I remember when It used to work, was that files wouldn't worked but I was thinking it was because lack of settings (because items added to library dont have settings), other than that I could list them by Title, Year, etc. Oh and I had issue with marking them as played that's why I was tweaking with countplay and later with studio.

The last time it worked was when I added the xbmc.log() to your error handler class. And it worked because each time I 'scanned' source it would throw those errors that '3' is not in empty list. But acording to what you wrote and what I tested It was long gone broken and yet Library worked because it was using old cached data, and would iterate over Library items and not the one from plugin.

@da3dsoul
Copy link
Contributor

da3dsoul commented Apr 5, 2019

That might help, actually

@da3dsoul
Copy link
Contributor

da3dsoul commented Apr 6, 2019

I think I got it. You can't map more than one episode to a single one by the numbers.

@da3dsoul
Copy link
Contributor

da3dsoul commented Apr 6, 2019

Stream info doesn't appear to be working, but that may be because I forgot to add it...IDK I've been trying a lot.

@da3dsoul
Copy link
Contributor

da3dsoul commented Apr 6, 2019

Fixed the stream info and cleaned it up. All that's left is checking for existence (cleaning DB) and maybe that hash thing.

@da3dsoul
Copy link
Contributor

da3dsoul commented Apr 6, 2019

Psh long term. How about now?

@da3dsoul
Copy link
Contributor

da3dsoul commented Apr 6, 2019

Okay, movies don't work at all. I'm working on extendedinfoscript quirks, but it's kind of working.

@da3dsoul
Copy link
Contributor

da3dsoul commented Apr 6, 2019

A way to improve it would be to get all images from the API, rather than the random ones we would get for the plugin menus.

@bigretromike
Copy link
Owner Author

bigretromike commented Apr 6, 2019 via email

@bigretromike
Copy link
Owner Author

bigretromike commented Apr 6, 2019 via email

@bigretromike
Copy link
Owner Author

That would make sense because we use playcount already. Thanks !

@bigretromike
Copy link
Owner Author

@AkariDN do you recall adding something to JSON-RPC to make files added from plugin source could be accessed by it ?
I'm sending this command:

{"jsonrpc":"2.0","method":"Files.GetFileDetails","params":{"file":"plugin://plugin.video.nakamori/tvshows/98939/play", "properties": ["episode", "title", "uniqueid", "tvshowid"]},"id":1}

I tried with file: plugin://plugin.video.nakamori/plugin://plugin.video.nakamori/tvshows/98939/play because in db the files are saved with as strPath = plugin://plugin.video.nakamori/tvshows/98939/play
while path linked to it is strPath = plugin://plugin.video.nakamori/ so I combine them, yet the kodi throw at me this kind of error:

2019-04-10 13:39:09.492 T:12520  NOTICE: {"jsonrpc":"2.0","method":"Files.GetFileDetails","params":{"file":"plugin://plugin.video.nakamori/tvshows/98939/play", "properties": ["episode", "title", "uniqueid", "tvshowid"]},"id":1}
2019-04-10 13:39:09.493 T:12520 WARNING: XFILE::CFileFactory::CreateLoader - unsupported protocol(plugin) in plugin://plugin.video.nakamori/tvshows/98939/play
2019-04-10 13:39:09.493 T:12520  NOTICE: {"error":{"code":-32602,"message":"Invalid params."},"id":1,"jsonrpc":"2.0"}

I print the command that I send that's why its there.

I'm not sure if this is not because the plugin was used as source so kodi now can't combine proper file path and searching by full_path_name is broken.

@AkariDN
Copy link

AkariDN commented Apr 10, 2019

Plugin URLs are not supported by this function:

JSONRPC_STATUS CFileOperations::GetFileDetails(const std::string &method, ITransportLayer *transport, IClient *client, const CVariant &parameterObject, CVariant &result)
{
  std::string file = parameterObject["file"].asString();
  if (!CFile::Exists(file))
    return InvalidParams;
...

@bigretromike
Copy link
Owner Author

Any chance you could make it work ? or any idea how to retrive db_id for item that I have full path to file that was added to db ?!
I would like to sync Library with Plugin for those who would like to use plugin on one Kodi and Library source on other.

@AkariDN
Copy link

AkariDN commented Apr 11, 2019

Unfortunately currently I have no time for this. Create an issue in Kodi repository, I hope somebody fix it.

I'm not an expert in Kodi jsonrpc, maybe there is some easier way except this:

  • Export your tvshows from plugin with 'tag' and 'uniqueid' properties (e.g. tag = Nakamori, uniqueid = tvshow/episode id)
  • Use VideoLibrary.GetTVShows and find 'tvshowid' by 'uniqueid'
    {"jsonrpc": "2.0", "id": 1, "method": "VideoLibrary.GetTVShows", "params": {"properties": ["uniqueid"], "filter": {"tag": "Nakamori"}}}
  • Use VideoLibrary.GetEpisodes and find 'episodeid' by 'uniqueid'
    {"jsonrpc": "2.0", "id": 1, "method": "VideoLibrary.GetEpisodes", "params": {"tvshowid": 1, "properties": ["uniqueid"]}}

@bigretromike
Copy link
Owner Author

bigretromike commented Apr 11, 2019 via email

@da3dsoul
Copy link
Contributor

Akari how dare you! Devs aren't allowed to have basic human needs, let alone lives! Do we need to take you to the White Room for educating?

Repository owner deleted a comment from da3dsoul Apr 11, 2019
@bigretromike
Copy link
Owner Author

I clean up as this is going to be linked.

I made and Issue: xbmc/xbmc#15897 not sure if I describe it the way it should be describe. Finger cross that someone pick that up and make fix for Leia.

@bigretromike
Copy link
Owner Author

@AkariDN hey, any idea why it throws this king of error for every tvshow that is being added to Video Library ?

2019-04-15 13:55:01.865 T:7140   ERROR: CScraperParser::Parse: Could not find scraper function GetArt
2019-04-15 13:55:01.865 T:7140   ERROR: ADDON::CScraper::Run: Unable to parse web site
2019-04-15 13:55:01.882 T:7140 WARNING: XFILE::CFileFactory::CreateLoader - unsupported protocol(plugin) in plugin://plugin.video.nakamori/tvshows/807

We use setArt() for our ListItems, but do we need to write some custom function GetArt or something?

@bigretromike
Copy link
Owner Author

Our lord and savior @AkariDN did push fix that end-up in kodi 18.2
While mine issues weren't resolved its a still nice thing to have.

without Files.GetFileDetails fix we can't sync watch from shoko to library after import it's still a stable thing.

So we having a great functionality for 18.2+

@da3dsoul
Copy link
Contributor

We might be able to with DB hacks similar to that watched state manager I've seen but never used

@bigretromike
Copy link
Owner Author

or if you are able to fix the shit inside kodi c++ code from issue I made :P and then I can make it work with proper API call ;p

@bigretromike
Copy link
Owner Author

Waiting for approval: xbmc/xbmc#16087
Until then this is really hard (or by somekind of funky/hacky workaround)

@bigretromike
Copy link
Owner Author

I'm still waiting for them to review my patch, I will announce this feature as part of 3.3.0 maybe we will have working sync between both.

@da3dsoul you said about some DB hacks, still got them ?

Im moving this to 3.3.0;

@bigretromike bigretromike added Kodi 19.x (Matrix) It involves Kodi 19 compatibility and removed Kodi 18.x (Leia) It involves Kodi 18 compatibility labels Sep 2, 2019
@bigretromike
Copy link
Owner Author

As this is possible in 18. (will probably need to add that in notes), I don't think that sync will be possible before 19.0, so moving this to 19.x

@bigretromike bigretromike added the Kodi 18.x (Leia) It involves Kodi 18 compatibility label Sep 3, 2019
@bigretromike bigretromike modified the milestones: long_term_goals, 3.5.0 Sep 3, 2019
@bigretromike
Copy link
Owner Author

Once again I would like to thank you @AkariDN - your work was a game changer for me :-) thanks.

Also:

  • Update Library work fine
  • Sync Plugin flags with VideoLibrary flags

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Kodi 18.x (Leia) It involves Kodi 18 compatibility Kodi 19.x (Matrix) It involves Kodi 19 compatibility New Feature A feature that will be part of Nakamori, but doesn't need new features from Shoko. Notes Notes for devs to keep ideas straight and/or remembered
Projects
None yet
Development

No branches or pull requests

3 participants