Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep protocol options in path databases #14168

Merged
merged 3 commits into from
Jul 13, 2018
Merged

Conversation

yol
Copy link
Member

@yol yol commented Jul 9, 2018

The code was inconsistent as to whether protocol options that may be present in source URIs are written to the paths table in the database.
Mostly there were two paths, one with and one without protocol options, leading to duplicated data.
This patch tries to fix this by settling on the version with protocol options at all times.

I hope this doesn't break too much other stuff, can't say really :-/

@yol yol requested review from mkortstiege and notspiff July 9, 2018 22:37
@yol yol self-assigned this Jul 9, 2018
@yol yol added Type: Fix non-breaking change which fixes an issue Component: Database v18 Leia labels Jul 9, 2018
@yol yol force-pushed the db-split-protocoloptions branch from 67f57ef to baf75bc Compare July 9, 2018 22:38
in addition to query options. Does not make much sense to remove the
query but keep protocol options.
This is needed for video/music database to function correctly as they
do not expect protocol options to turn up in the filename.
URIUtils::Split now removes protocol options, and not adding them
to the path would cause them to get lost when the path from the
database is used directly.
URIUtils::Split now removes protocol options, and not adding them
to the path would cause them to get lost when the path from the
database is used directly.
@yol yol force-pushed the db-split-protocoloptions branch from baf75bc to 1edd0f1 Compare July 9, 2018 22:40
Copy link
Contributor

@notspiff notspiff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code is fine, but obviously I cannot say anything about the regressions either. We just have to try.

@DaveTBlake
Copy link
Member

This in effect reverses the change made by @koying in #12962 (Only remove options for actual URL in URIUtils::Split ) that was done to fix Trac ticket https://trac.kodi.tv/ticket/17627

So can we discuss what we are doing with stored path and whether it has options or not before we "just try". That PR broke other things that have been fixed but took a while to find

@fritsch
Copy link
Member

fritsch commented Jul 11, 2018

"Just try" is the only option to find and spot the 100 000 of workarounds already in the codebase to have it sane once and for all.

@DaveTBlake
Copy link
Member

If the consensus (from discussion on Slack) is that URL options be stored - as after all where else are they going to come from - and that #12962 was a mistake, then wouldn't a better solution be to restore how URIUtils::Split worked and fix the related ticket another way?

@yol
Copy link
Member Author

yol commented Jul 11, 2018

This in effect reverses the change made by @koying in #12962

I don't really see this connection - why do you think it reverses #12962? Test cases still work (I should add a new one to cover this change though)

@DaveTBlake
Copy link
Member

Before #12962 we were storing options in path, hence this change and that PR seem closely related to me. I just wanted everyone to be aware of the related previous change and issues when reviewing this.

If we want to do store options in path again, and that does seem reasonable (but with more testing than just running the limited test cases with have) , then why not do it in URIUtils::Split instead of adding extra method?

@yol
Copy link
Member Author

yol commented Jul 11, 2018

Before #12962 we were storing options in path, hence this change and that PR seem closely related to me. I just wanted everyone to be aware of the related previous change and issues when reviewing this.

#12962 seems completely unrelated to protocol options to me. Are you maybe confusing query options (http://localhost/a?b=c) and protocol options (http://localhost/a|b=c)?
Anyway, as far as I understood it the problem for #12962 was not about storing options at all, but that a ? in filesystem paths (not URLs) was wrongly interpreted as query option and thus discarded - because query options were not stored (they still are not).

@DaveTBlake
Copy link
Member

Yes I am lumping all options (query + protocol) together, sorry if insufficiently nuanced. Having two PR changing what gets stored in the databases (one more obviously than the other), and both questioning if there could be regressions etc., was enough for me to want it at least considered together.

It would help greatly if we had a documented db design, rather than just colum name, to go on (for example does path include options or not, what kind or where else are they stored). But we are where we are, and Spiff's memory is the great asset (if only we could use Google to search it)

If you have looked at the other PR then fair enough.

@yol
Copy link
Member Author

yol commented Jul 11, 2018

Yeah as far as I can see the other PR is orthogonal. Thanks for bringing it up though, really difficult to keep all of this stuff in mind.

@yol
Copy link
Member Author

yol commented Jul 13, 2018

Guess we'll just have to try this, @MartijnKaijser said he's fine with it

@yol yol merged commit 34eaf78 into xbmc:master Jul 13, 2018
@yol yol added this to the Leia 18.0-alpha3 milestone Jul 13, 2018
@FernetMenta
Copy link
Contributor

I would have chosen the opposite: always strip options. Note that path is actually the ID of an item. With this change all info of an item is lost if some options are changed.

@yol
Copy link
Member Author

yol commented Jul 13, 2018

I would have chosen the opposite: always strip options.

Then where do you get them from when playing an item?

With this change all info of an item is lost if some options are changed.

Yes. Same as if you would change the path. Certainly not nice, but do you know a way around it? Theoretically we could try to put protocol options into an extra column. Not sure it's worth the trouble though.

@DaveTBlake
Copy link
Member

I don't see a problem in storing the protocol options (or any others) in the path string. However we do need to ensure that when file item processing is trying to match items it only uses the appropriate part of this full path.

@yol yol deleted the db-split-protocoloptions branch November 19, 2018 08:49
yol referenced this pull request Nov 19, 2018
Not verifying any certificates renders SSL meaningless. cUrl should have
a stock of default CA certificates provided by the system/distributor.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Database Type: Fix non-breaking change which fixes an issue v18 Leia
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants