Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Get rid of LIKE clauses for path matching as mysql has issues with slash escaping otherwise #804

Merged
merged 4 commits into from Apr 5, 2012

Conversation

Projects
None yet
3 participants
Member

jmarshallnz commented Mar 23, 2012

When presented with a LIKE clause, mysql first parses the string, which treats any backslashes as escape characters. This means any DOS paths will screw up: They'll be correctly added to the database, but a LIKE clause on the exact same string will not match due to the escaping.

The simple solution is to use = clauses instead of LIKE clauses. This is easy in most cases as we match on exact paths.

Where it isn't quite so easy is when we want to match subpaths. We do this by employing the SUBSTR() function, which takes in utf8 character lengths, rather than byte lengths.

Note that this hasn't as yet been fully tested on win32 mysql (earlier versions of the same patch have been tested) so if anyone happens to have such a setup, please test.

Contributor

xbmcfanboy commented Apr 5, 2012

Been testing this for about an hour with no problems. I'm a Windows user with mysql library centrallized on NAS. It appears to work fine for both SQLite and mysql. Example of the queries it produced for me. I hope this helps.

SQLite
"select * from view where window = 10501 and path='?/' and skin='skin.confluence'"

mysql (sanitized)
"select idPath from path where strPath='smb://myusername:mypassword@ipaddress/Multimedia/Movies/'"

Member

jmarshallnz commented Apr 5, 2012

What really needs testing is whether win32 paths work. i.e. you should be able to mount your NAS as a windows drive letter and then mysql should still work that that as well.

This affects only the video + music libraries using mysql, ofcourse.

Contributor

xbmcfanboy commented Apr 5, 2012

Added source U:, opened the source, right-clicked, chose a scraper and type, then clicked OK. Then re-opened set content to verify it saved the scraper settings. I also ran the mysql queries manually in phpMyAdmin to see if any had syntax errors.

SQLite results
"select * from view where window = 10501 and path='U:\Movies\The Addams Family (1991)\' and skin='skin.confluence'"
"select * from view where window = 10501 and path='U:\Movies\The Addams Family (1991)\'"

MySQL results
"select idPath from path where strPath='U:\\Movies\\The Addams Family (1991)\\'"
"SELECT idPath,strPath FROM path WHERE SUBSTR(strPath,1,35)='U:\\Movies\\The Addams Family (1991)\\'"
"select files.strFilename from files join movie on movie.idFile=files.idFile where files.idPath=594"
"select files.strFilename from files join musicvideo on musicvideo.idFile=files.idFile where files.idPath=594"
"update path set strContent='', strScraper='', strHash='',strSettings='',useFolderNames=0,scanRecursive=0 where idPath=594"
"select strHash from path where strPath='U:\\Movies\\The Addams Family (1991)\\'"
"select idPath from path where strPath='U:\\Movies\\The Addams Family (1991)\\.actors\\'"
Member

jmarshallnz commented Apr 5, 2012

Thanks :) Looks all good - really appreciate the testing.

@jmarshallnz jmarshallnz was assigned Apr 5, 2012

@jmarshallnz jmarshallnz added a commit that referenced this pull request Apr 5, 2012

@jmarshallnz jmarshallnz Merge pull request #804 from jmarshallnz/i_dont_like_paths
Get rid of LIKE clauses for path matching as mysql has issues with slash escaping otherwise
03e0857

@jmarshallnz jmarshallnz merged commit 03e0857 into xbmc:master Apr 5, 2012

pieh commented on d9ebf0f May 11, 2012

lately I run into problems in few more queries that propably need same treatment: https://gist.github.com/178db5cf98183995f248 (stacks will have stack://fullpath1, fullpath2 in strFileName column so stacks on local win32 paths will still be buggy because of 'LIKE' instead of '=')

Owner

jmarshallnz replied May 11, 2012

Yup - I have local fixes for that case (and rar/zip)

@HolgerW1 HolgerW1 pushed a commit to HolgerW1/xbmc that referenced this pull request Sep 26, 2014

Patrick Vos Merge branch 'change_SB-API_cleanup_PR-804-thezoggy' into development
* change_SB-API_cleanup_PR-804-thezoggy:  closes gh-804
   SB-API: PEP8 fixes, etree import correction, use unicode not string when returning show name (show.refresh/show.pause/show.update).
   APIBuilder: Limit show list to 100 shows to ensure performance (since the list is used 9 times for the linked list dropdown)
be1d1c5

@HolgerW1 HolgerW1 pushed a commit to HolgerW1/xbmc that referenced this pull request Sep 26, 2014

Patrick Vos Merge branch 'development'
Add require/ignore words per show. Pull #799
Some notable examples below of using ignore/require words,
also the global ignore words in the webui under search settings.
Ignore webrip or 'xvid' while still getting web-dl.
Require a specific's groups rip, like QRUS or ignore one like mSD.
Grab HD rips but require it to have DD5 audio.
Require VFR (variable frame rate) for the show Top Gear.
Require Uncut or XL versions of a show (since TVDB does not allow 'duplicate' shows).
Do note that the ignore word logic comes before required words. So be careful on what you ignore.
The words are not case sensitive, and if you use multiple words then a release will be rejected if ANY match.

Fix bug on massEdit removing values not displayed (air-by-date, language, release ignore/require words).

Remove superfluous logging message when editing show.

Do not set a show to air-by-date if it contains Documentary -- fixes Top Gear

Update scene_exception url to point new github domain. Pull #810

Change PostProcessing checks and rules. Pull #803
Ignore subfolders that start with . instead of a specific ignore list.
Pass (inherit) nzbName to subfolder
Add postProcessor checks: status, quality/filesize, already processed.
Change use quality from snatch history instead of status quality
Delete empty folder (scripts and automatic modes, not manual).
Change search for airdate in database instead of thetvdb.
Includes PR-620-jayme-github Process files by size but reversed (biggest first).
Stop postprocessing when failing to create destination folder.

Update autoProcessTV to work in python 2 and 3 (note that sickbeard itself only works in 2.5+ and not 3). Pull #802

API and NZBGet cleanup. Pull #804
Updated NZBGet to allow setting username (requires NZBGet v11+).
Updated NZBGet to have real priority for (requires NZBGet v9+) instead of just adding to the top of the queue.
Change NZBGet host field to an url so that people can specify the protocol (http/https).

SB-API: PEP8 fixes, use unicode not string when returning show name for show.refresh / show.pause / show.update.
APIBuilder: Limit show list to 100 shows to ensure performance
(since the show list is used multiple times for different linked list)

Lower default RSS search freq. from 60 min to 40 mins.

Upgrade Jquery UI from 1.10.3 to 1.10.4. Pull #793

Quick pass over our .js to cleanup whitespace, add missing semicolons, url redirect fixes. Pull #795

Handle malformed tvshow.nfo files when using the Add Existing Show page and resolved
the loading indicator to not finish.

Fix Exception on adding the initial rootdir.

Add routines to check or clean config input from the GUI.

Change usenet-crawler to https

Use with openwhen opening/writing files so it will automatically close the file,
even if there is an exception. Applied whitespace PEP8 fixes to the files touched,
also remove obsolete functions using open.

Fix AttributeError exception for metadata if stripping genre/actor name when None. Pull #801

Torrent provider HDbits now supports backlog and proper searches.

Metadata: Mede8er added.
f64b94f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment