Tests and fixes (#8124)

* Replace feedparser with BS4Parser Signed-off-by: miigotu <miigotu@gmail.com> * Remove last reference to feedparser Signed-off-by: miigotu <miigotu@gmail.com> * Allow specifying a custom list of sort articles, useful for users of foreign languages. Default is the|a|an, but for example spanish would be "un|una|el|la|los|las" Can be set in config->general or under the Localization domain in the config.ini Fixes #8081 Signed-off-by: miigotu <miigotu@gmail.com> * Clean up regex for tracker matcher and double space in printing parser result for anime Signed-off-by: miigotu <miigotu@gmail.com> * Update bs4_parser.py * Fix some grammar warnings in processTV before I make any real changes Signed-off-by: miigotu <miigotu@gmail.com> * Remove meta_path hack and poetry_install code. Im sick of the bugs and hackiness for it. Guides need updated to instruct people how to install dependencies! Signed-off-by: miigotu <miigotu@gmail.com> * When validating the dir for a post process, or actually processing a file, first parse it with guessit and see if we can find the show to match it to. If so, we know to only use anime regexes or regular regexes. Fallback to old way otherwise. Fix up some logging in post processor and processTV Signed-off-by: miigotu <miigotu@gmail.com> * Remove necessity for version.py, so it cant be out of sync anymore Signed-off-by: miigotu <miigotu@gmail.com> * Update to new versions of beekeeper-alt and kodipydent-alt to remove odd poetry-date-version-plugin references from their source Signed-off-by: miigotu <miigotu@gmail.com> * Fix formatting with black and isort Signed-off-by: miigotu <miigotu@gmail.com> * Fix nyaa parsing Signed-off-by: miigotu <miigotu@gmail.com> * try_int for seeders and leechers in nyaa Signed-off-by: miigotu <miigotu@gmail.com> * Not Snatched change Warning to Info. * Update pyproject.toml * Update pythonpackage.yml * Update upgrade_dependencies.yml * Update release.yml * Remove old newznab provider Signed-off-by: miigotu <miigotu@gmail.com> * Extract parsing xml items into a reusable mixin, fixes parsing size in rss searches Signed-off-by: miigotu <miigotu@gmail.com> * Rename newnewznab provider file to newznab.py, ready for prime time I think Signed-off-by: miigotu <miigotu@gmail.com> * Fix torznab/jackett and others parsing with parse_rss_item. Signed-off-by: miigotu <miigotu@gmail.com> Signed-off-by: miigotu <miigotu@gmail.com> Co-authored-by: BKSteve <stephen.rieger@gmail.com>
SickChill · Sep 5, 2022 · 4c50ad3 · 4c50ad3
1 parent 57da5c4
commit 4c50ad3
Show file tree

Hide file tree

Showing 50 changed files with 519 additions and 1,142 deletions.
diff --git a/.github/workflows/pythonpackage.yml b/.github/workflows/pythonpackage.yml
@@ -94,7 +94,7 @@ jobs:
     - name: Install Poetry
       uses: snok/install-poetry@v1.3.1
       with:
-        version: 1.1.14
+        version: 1.2.0
         virtualenvs-path: .venv
         virtualenvs-create: true
         virtualenvs-in-project: true
@@ -177,7 +177,7 @@ jobs:
     - name: Install Poetry
       uses: snok/install-poetry@v1.3.1
       with:
-        version: 1.1.14
+        version: 1.2.0
         virtualenvs-create: false
         installer-parallel: false
     - name: Docker meta

diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
@@ -15,7 +15,7 @@ jobs:
       - name: Install Poetry
         uses: snok/install-poetry@v1.3.1
         with:
-          version: 1.1.14
+          version: 1.2.0
           virtualenvs-path: .venv
           virtualenvs-create: true
           virtualenvs-in-project: true

diff --git a/.github/workflows/upgrade_dependencies.yml b/.github/workflows/upgrade_dependencies.yml
@@ -46,7 +46,7 @@ jobs:
       - name: Install Poetry
         uses: snok/install-poetry@v1.3.1
         with:
-          version: 1.1.14
+          version: 1.2.0
           virtualenvs-path: .venv
           virtualenvs-create: true
           virtualenvs-in-project: true

diff --git a/Gruntfile.js b/Gruntfile.js
@@ -71,7 +71,7 @@ module.exports = function(grunt) {
 
         // Start merging and releasing
         'exec:git:merge:develop --strategy-option theirs', // Merge develop into master
-        'exec:bump_version', // Update version.py
+        'exec:bump_version', // Update version in pyproject.toml
         'exec:commit_changed_files:yes', // Commit the new changed version
         'exec:git_list_changes', // List changes from since last tag
         'exec:git_tag_next_version', // Create new release tag
@@ -286,16 +286,6 @@ module.exports = function(grunt) {
                     if (!stdout.match(/Bumping version from \d{4}\.\d{1,2}\.\d{1,2}(\.\d*)?/)) {
                         grunt.fatal('Did the version update in pyproject.toml?')
                     }
-                    const next_version = grunt.config('next_version');
-                    const file = 'sickchill/version.py';
-                    grunt.log.ok('Updating ' + file + ' to ' + next_version);
-
-                    if (!grunt.file.exists(file)) {
-                        grunt.log.error('Could not find ' + file + ', cannot proceed');
-                    }
-
-                    grunt.file.delete(file);
-                    grunt.file.write(file, '__version__ = "' + next_version + '"');
                 }
             },
 
@@ -320,7 +310,7 @@ module.exports = function(grunt) {
             'commit_changed_files': { // Choose what to commit.
                 cmd: function(ci) {
                     grunt.config('stop_no_changes', Boolean(ci));
-                    return 'git status -s -- pyproject.toml sickchill/version.py sickchill/locale/ sickchill/gui/';
+                    return 'git status -s -- pyproject.toml sickchill/locale/ sickchill/gui/';
                 },
                 stdout: false,
                 callback: function(err, stdout) {
@@ -341,7 +331,6 @@ module.exports = function(grunt) {
                             grunt.fatal('Tried to commit a version change without updating the version in both places!')
                         }
                         commitMsg.push('Release version ' + grunt.config('next_version'));
-                        commitPaths.push('sickchill/version.py');
                         commitPaths.push('pyproject.toml');
                     }
                     if (stdout.match(/sickchill\/gui\/.*(vendor|core)\.min\.(js|css)$/gm)) {

diff --git a/pyproject.toml b/pyproject.toml
@@ -40,14 +40,43 @@ packages = [
 [tool.poetry.scripts]
 SickChill = "SickChill:main"
 
+# The way poetry reads dependency groups from pyproject.toml changed between 1.1.4 and 1.2.0
+# Going to try the old tag with 1.2.0 until more people have peotry 1.2.0+ installed.
+#[tool.poetry.group.dev.dependencies]
+[tool.poetry.dev-dependencies]
+black = "^22.8.0"
+coveralls = "^3.3.1"
+mock = "^4.0.3"
+Babel = "^2.9.0"
+pytest-cov = "^3.0.0"
+pytest-isort = "^3.0.0"
+flake8-pytest-style = "^1.5.1"
+flake8-commas = "^2.0.0"
+flake8-quotes = "^3.3.0"
+flake8-bandit = "^2.1.2"
+flake8-docstrings = "^1.6.0"
+flake9 = "^3.8.3"
+codecov = "^2.1.12"
+pytest = "^7.0.1"
+isort = "^5.10.1"
+poethepoet = "^0.16.0"
+po2json = "^0.2.2"
+bandit = "<=1.7.2"
+pyaes = {version = "^1.6.1", optional = true}
+types-attrs = {version = "^19.1.0", optional = true}
+types-pyOpenSSL = {version = "^22.0.10", optional = true}
+types-python-dateutil = {version = "^2.8.19", optional = true}
+types-python-slugify = {version = "^6.1.0", optional = true}
+types-requests = {version = "^2.28.6", optional = true}
+types-chardet = {version = "^5.0.3", optional = true}
+types-pytz = {version = "^2022.1.2", optional = true}
+types-setuptools = {version = "^65.1.0", optional = true}
+types-six = {version = "^1.16.18", optional = true}
+
 [build-system]
-requires = ["poetry-core>=1.0.4"]
+requires = ["poetry-core"]
 build-backend = "poetry.core.masonry.api"
 
-[tool.version-plugin]
-write_to = 'sickchill/version.py'
-regex = '__version__ = "{version}"'
-
 [tool.flake8]
 exclude = [
     ".eggs",
@@ -191,11 +220,10 @@ pybabel update
 """
 
 [tool.poetry.dependencies]
-python = ">=3.7,<4"
+python = ">=3.7.2,<4"
 "bencode.py" = "^4.0.0"
 CacheControl = "^0.12.6"
 configobj = "^5.0.6"
-feedparser = "^6.0.2"
 greenlet = {version = ">=1.1.2", allow-prereleases = true}
 html5lib = {version = "^1.1", optional = true}
 ipaddress = "^1.0.23"
@@ -238,35 +266,5 @@ guessit = "^3.3.1"
 requests = "^2.25.1"
 IMDbPY = ">=2020.09.25"
 babelfish = ">=0.6.0b1"
-kodipydent-alt = "^2021.7.16"
-beekeeper-alt = "^2021.7.16"
-
-[tool.poetry.dev-dependencies]
-coveralls = "^3.3.1"
-mock = "^4.0.3"
-Babel = "^2.9.0"
-pytest-cov = "^3.0.0"
-pytest-isort = "^3.0.0"
-flake8-pytest-style = "^1.5.1"
-flake8-commas = "^2.0.0"
-flake8-quotes = "^3.3.0"
-flake8-bandit = "^2.1.2"
-flake8-docstrings = "^1.6.0"
-flake9 = "^3.8.3"
-codecov = "^2.1.12"
-black = "^22.1"
-pytest = "^7.0.1"
-isort = "^5.10.1"
-poethepoet = "^0.16.0"
-po2json = "^0.2.2"
-bandit = "<=1.7.2"
-pyaes = {version = "^1.6.1", optional = true}
-types-attrs = {version = "^19.1.0", optional = true}
-types-pyOpenSSL = {version = "^22.0.10", optional = true}
-types-python-dateutil = {version = "^2.8.19", optional = true}
-types-python-slugify = {version = "^6.1.0", optional = true}
-types-requests = {version = "^2.28.6", optional = true}
-types-chardet = {version = "^5.0.3", optional = true}
-types-pytz = {version = "^2022.1.2", optional = true}
-types-setuptools = {version = "^65.1.0", optional = true}
-types-six = {version = "^1.16.18", optional = true}
+kodipydent-alt = ">=2022.9.3"
+beekeeper-alt = ">=2022.9.3"
diff --git a/sickchill/__init__.py b/sickchill/__init__.py
@@ -1,6 +1,5 @@
-from sickchill.init_helpers import maybe_daemonize, poetry_install
+from sickchill.init_helpers import maybe_daemonize
 
 maybe_daemonize()
-# poetry_install()
 
 from .show.indexers import indexer, ShowIndexer
diff --git a/sickchill/gui/slick/js/core.js b/sickchill/gui/slick/js/core.js
@@ -3,7 +3,7 @@ function getMeta(pyVar) {
 }
 
 const scRoot = getMeta('scRoot');
-const srDefaultPage = getMeta('srDefaultPage');
+const scDefaultPage = getMeta('scDefaultPage');
 const themeSpinner = getMeta('themeSpinner');
 const anonURL = getMeta('anonURL');
 const topImageHtml = '<img src="' + scRoot + '/images/top.gif" width="31" height="11" alt="Jump to top" />'; // eslint-disable-line no-unused-vars
@@ -2423,7 +2423,8 @@ const SICKCHILL = {
                     getSortData: {
                         name(itemElement) {
                             const name = $(itemElement).attr('data-name') || '';
-                            return latinize((metaToBool('settings.SORT_ARTICLE') ? name : name.replace(/^((?:the|a|an)\s)/i, '')).toLowerCase().normalize('NFC'));
+                            const regex = new RegExp('^((?:' + getMeta('settings.GRAMMAR_ARTICLES') + ')\\s)', 'i');
+                            return latinize((metaToBool('settings.SORT_ARTICLE') ? name : name.replace(regex, '')).toLowerCase().normalize('NFC'));
                         },
                         network: '[data-network]',
                         date(itemElement) {
@@ -2554,7 +2555,8 @@ const SICKCHILL = {
                     getSortData: {
                         name(itemElement) {
                             const name = $(itemElement).attr('data-name') || '';
-                            return latinize((metaToBool('settings.SORT_ARTICLE') ? name : name.replace(/^((?:the|a|an)\s)/i, '')).toLowerCase().normalize('NFC'));
+                            const regex = new RegExp('^((?:' + getMeta('settings.GRAMMAR_ARTICLES') + ')\\s)', 'i');
+                            return latinize((metaToBool('settings.SORT_ARTICLE') ? name : name.replace(regex, '')).toLowerCase().normalize('NFC'));
                         },
                         network: '[data-network]',
                         date(itemElement) {
@@ -2586,7 +2588,8 @@ const SICKCHILL = {
                     getSortData: {
                         name(itemElement) {
                             const name = $(itemElement).attr('data-name') || '';
-                            return latinize((metaToBool('settings.SORT_ARTICLE') ? name : name.replace(/^((?:the|a|an)\s)/i, '')).toLowerCase().normalize('NFC'));
+                            const regex = new RegExp('^((?:' + getMeta('settings.GRAMMAR_ARTICLES') + ')\\s)', 'i');
+                            return latinize((metaToBool('settings.SORT_ARTICLE') ? name : name.replace(regex, '')).toLowerCase().normalize('NFC'));
                         },
                         network: '[data-network]',
                         date(itemElement) {
@@ -3228,7 +3231,7 @@ const SICKCHILL = {
                                 $.post(scRoot + '/home/is-alive/', () => { // eslint-disable-line max-nested-callbacks
                                     clearInterval(checkIsAlive);
                                     setTimeout(() => { // eslint-disable-line max-nested-callbacks
-                                        window.location = scRoot + '/' + srDefaultPage + '/';
+                                        window.location = scRoot + '/' + scDefaultPage + '/';
                                     }, 3000);
                                 }, 'jsonp');
                             }, 1000);
@@ -4035,7 +4038,8 @@ const SICKCHILL = {
                     getSortData: {
                         name(itemElement) {
                             const name = $(itemElement).attr('data-name') || '';
-                            return (metaToBool('settings.SORT_ARTICLE') ? name : name.replace(/^((?:the|a|an)\s)/i, '')).toLowerCase();
+                            const regex = new RegExp('^((?:' + getMeta('settings.GRAMMAR_ARTICLES') + ')\\s)', 'i');
+                            return (metaToBool('settings.SORT_ARTICLE') ? name : name.replace(regex, '')).toLowerCase();
                         },
                         rating: '[data-rating] parseInt',
                         votes: '[data-votes] parseInt',

diff --git a/sickchill/gui/slick/js/core.min.js b/sickchill/gui/slick/js/core.min.js
diff --git a/sickchill/gui/slick/js/parsers.js b/sickchill/gui/slick/js/parsers.js
@@ -8,7 +8,8 @@ $.tablesorter.addParser({
             return s.replace(_('Loading...'), '000');
         }
 
-        return latinize(metaToBool('settings.SORT_ARTICLE') ? (s || '') : (s || '').replace(/^(the|a|an)\s/i, '').normalize('NFC'));
+        const regex = new RegExp('^((?:' + getMeta('settings.GRAMMAR_ARTICLES') + ')\\s)', 'i');
+        return latinize(metaToBool('settings.SORT_ARTICLE') ? (s || '') : (s || '').replace(regex, '').normalize('NFC'));
     },
     type: 'text',
 });

diff --git a/sickchill/gui/slick/js/trendingShows.js b/sickchill/gui/slick/js/trendingShows.js
@@ -12,7 +12,8 @@ $(document).ready(() => {
             getSortData: {
                 name(itemElement) {
                     const name = $(itemElement).attr('data-name') || '';
-                    return (metaToBool('settings.SORT_ARTICLE') ? name : name.replace(/^(the|a|an)\s/i, '')).toLowerCase();
+                    const regex = new RegExp('^((?:' + getMeta('settings.GRAMMAR_ARTICLES') + ')\\s)', 'i');
+                    return (metaToBool('settings.SORT_ARTICLE') ? name : name.replace(regex, '')).toLowerCase();
                 },
                 rating: '[data-rating] parseInt',
                 votes: '[data-votes] parseInt',

diff --git a/sickchill/gui/slick/views/addShows_favoriteShows.mako b/sickchill/gui/slick/views/addShows_favoriteShows.mako
@@ -6,6 +6,7 @@
 %>
 <%block name="metas">
     <meta data-var="settings.SORT_ARTICLE" data-content="${settings.SORT_ARTICLE}">
+    <meta data-var="settings.GRAMMAR_ARTICLES" data-content="${settings.GRAMMAR_ARTICLES}">
 </%block>
 <%block name="content">
     <div class="col-md-12">

diff --git a/sickchill/gui/slick/views/addShows_popularShows.mako b/sickchill/gui/slick/views/addShows_popularShows.mako
@@ -8,6 +8,7 @@
 %>
 <%block name="metas">
     <meta data-var="settings.SORT_ARTICLE" data-content="${settings.SORT_ARTICLE}">
+    <meta data-var="settings.GRAMMAR_ARTICLES" data-content="${settings.GRAMMAR_ARTICLES}">
 </%block>
 <%block name="content">
     <div class="col-md-12">

diff --git a/sickchill/gui/slick/views/config_general.mako b/sickchill/gui/slick/views/config_general.mako
@@ -520,6 +520,16 @@
                             </div>
                         </div>
 
+                        <div class="field-pair row">
+                            <div class="col-lg-3 col-md-4 col-sm-5 col-xs-12">
+                                <label class="component-title">${_('Definite/Indefinite articles to remove for sorting')}</label>
+                            </div>
+                            <div class="col-lg-9 col-md-8 col-sm-7 col-xs-12 component-desc">
+                                <input type="text" name="grammar_articles" id="grammar_articles" value="${settings.GRAMMAR_ARTICLES}" class="form-control input-sm input100" autocapitalize="off" />
+                                <label for="grammar_articles">${_('List of words to treat as articles, separated by the pipe character. Default: "the|a|an"')}</label>
+                            </div>
+                        </div>
+
                         <div class="field-pair row">
                             <div class="col-lg-3 col-md-4 col-sm-5 col-xs-12">
                                 <label class="component-title">${_('Missed episodes range')}</label>

diff --git a/sickchill/gui/slick/views/inc_home_show_list_poster.mako b/sickchill/gui/slick/views/inc_home_show_list_poster.mako
@@ -66,6 +66,8 @@
                 if curShow.paused:
                     display_status = _(display_status) + ' ' + _('Paused')
 
+                cur_airs_next = None
+                cur_airs_prev = None
                 if curShow.indexerid in show_stat:
                     cur_airs_next = show_stat[curShow.indexerid]['ep_airs_next']
                     cur_airs_prev = show_stat[curShow.indexerid]['ep_airs_prev']

diff --git a/sickchill/gui/slick/views/layouts/main.mako b/sickchill/gui/slick/views/layouts/main.mako
@@ -61,6 +61,7 @@
         <meta data-var="settings.POSTER_SORTDIR" data-content="${settings.POSTER_SORTDIR}">
         <meta data-var="settings.ROOT_DIRS" data-content="${settings.ROOT_DIRS}">
         <meta data-var="settings.SORT_ARTICLE" data-content="${settings.SORT_ARTICLE}">
+        <meta data-var="settings.GRAMMAR_ARTICLES" data-content="${settings.GRAMMAR_ARTICLES}">
         <meta data-var="settings.TIME_PRESET" data-content="${settings.TIME_PRESET}">
         <meta data-var="settings.TRIM_ZERO" data-content="${settings.TRIM_ZERO}">
         <meta data-var="settings.SICKCHILL_BACKGROUND" data-content="${settings.SICKCHILL_BACKGROUND}">

diff --git a/sickchill/gui/slick/views/restart.mako b/sickchill/gui/slick/views/restart.mako
@@ -1,6 +1,6 @@
 <%inherit file="/layouts/main.mako"/>
 <%block name="metas">
-    <meta data-var="srDefaultPage" data-content="${sbDefaultPage}">
+    <meta data-var="scDefaultPage" data-content="${sbDefaultPage}">
 </%block>
 
 <%block name="css">

diff --git a/sickchill/gui/slick/views/trendingShows.mako b/sickchill/gui/slick/views/trendingShows.mako
@@ -5,6 +5,7 @@
 %>
 <%block name="metas">
     <meta data-var="settings.SORT_ARTICLE" data-content="${settings.SORT_ARTICLE}">
+    <meta data-var="settings.GRAMMAR_ARTICLES" data-content="${settings.GRAMMAR_ARTICLES}">
 </%block>
 <%block name="scripts">
     <script type="text/javascript" src="${static_url('js/trendingShows.js')}"></script>