python3 support ?

[m040601]

Hi, thanks for your work in this interesting fork.

I'm having the same problem as everyone with the KeyError: u'sort_id'
#14

I'm on ArchLinux, and the offilicial (and only) python version is python3.
Theres is also calibre for python 3 and calibre for python 2.

I tried every possible combination of python versions and calibre, but I still get that error, when trying to run ebook-convert with your recipe.

Anyway, since python2 is a thing of the past, is there any reason
your recipe couldnt work with python3 ?

[mmagnus]

I think it should work with Python3, I remove the Python specification from the recipe.
I hope it will work. Please try and let me know.

Hmm.. in my case I run Calibre in the debug-mode, I still see Python2.7.16, I'm not sure how this Python is selected?

(py37) [mx] cwc2_trim$ calibre-debug --paths --gui-debug xx
calibre Debug log
calibre 4.13  embedded-python: True is64bit: True
Darwin-19.4.0-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '19.4.0', 'Darwin Kernel Version 19.4.0: Wed Mar  4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64')
Python 2.7.16
OSX: ('10.15.4', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: Goodreads Sync (1, 14, 2)
calibre 4.13  embedded-python: True is64bit: True
Darwin-19.4.0-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '19.4.0', 'Darwin Kernel Version 19.4.0: Wed Mar  4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64')
Python 2.7.16
OSX: ('10.15.4', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: Goodreads Sync (1, 14, 2)
devicePixelRatio: 2.0
logicalDpi: 72.0 x 72.0
physicalDpi: 109.000001637 x 109.000001637
Using calibre Qt style: True
[0.00] Starting up...
[0.02] Showing splash screen...
[0.40] splash screen shown
[0.40] Initializing db...
[0.48] db initialized
[0.48] Constructing main UI...
DEBUG:    0.0 Goodreads Sync::dialogs.py - loading translations
DEBUG:    0.0 GoodreadsSync::action.py - loading translations
sys.version_info(major=2, minor=7, micro=16, releaselevel='final', serial=0)

[m040601]

I think it should work with Python3,
I remove the Python specification from the recipe.
I hope it will work.

Please clarify, because I'm not 100% sure what you mean.
So what you mean, is that you yourself run it with python2 installed on your system right ? You dont' have a testing environment with only python 3 ?

Please try and let me know.

This is to continue giving you feedback of trying to run this recipe in a system with only python3.

As I told before we have 3 versions available in ArchLinux:

community/calibre 4.16.0-1 [installed]
Ebook management application (python2 build)
community/calibre-common 4.16.0-1 [installed]
Ebook management application (common files)
community/calibre-python3 4.16.0-1
Ebook management application (experimental python3 port)

calibre-common has to be installed always, no matter what calibre version you choose.
If I install only calibre-python3, this recipe fails frequently with

TypeError: 'dict_keys' object is not subscriptable
I ran it with this configuration:

# CONFIGURATION ###########################################################         
TAGS = [] # [] or ['tag1', 'tag2']                                                  
TAGS_EXCEPTIONS = [] # [] or ['tag3', 'tag4']                                       
INCLUDE_UNTAGGED = True                                                             
# ARCHIVE_DOWNLOADED = True                                                         
ARCHIVE_DOWNLOADED = False                                                          
MAX_ARTICLES_PER_FEED = 30                                                          
# OLDEST_ARTICLE = 7                                                                
OLDEST_ARTICLE = 30                                                                 
SORT_METHOD = 'newest'                                                              
TO_PULL = 'unread'                                                                  
TITLE_WITH_TAGS = False                                                             
#############################################################################                   

Output of running

ebook-convert ./magnus.recipe myebook.epub --username=my@email.com --password=mypocketpassword --test -vv --debug-pipeline debug
I do not use calibre the GUI. I'm only interested in the "ebook-convert" cli tool provided by calibre.

1% Converting input to HTML...
InputFormatPlugin: Recipe Input running
Using custom recipe
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36
1% Fetching feeds...
Traceback (most recent call last):
  File "/usr/bin/ebook-convert", line 20, in <module>
    sys.exit(main())
  File "/usr/lib/calibre/calibre/ebooks/conversion/cli.py", line 401, in main
    plumber.run()
  File "/usr/lib/calibre/calibre/ebooks/conversion/plumber.py", line 1108, in run
    self.oeb = self.input_plugin(stream, self.opts,
  File "/usr/lib/calibre/calibre/customize/conversion.py", line 245, in __call__
    ret = self.convert(stream, options, file_ext,
  File "/usr/lib/calibre/calibre/ebooks/conversion/plugins/recipe_input.py", line 139, in convert
    ro.download()
  File "/usr/lib/calibre/calibre/web/feeds/news.py", line 1028, in download
    res = self.build_index()
  File "/usr/lib/calibre/calibre/web/feeds/news.py", line 1196, in build_index
    feeds = feeds_from_index(self.parse_index(), oldest_article=self.oldest_article,
  File "<string>", line 292, in parse_index
TypeError: 'dict_keys' object is not subscriptable

Strangely if I change
OLDEST_ARTICLE = 30 back to the default
OLDEST_ARTICLE = 7 , it sometimes works

If I then remove from my system calibre-common and calibre-python3 and instal calibre (python2) and reinstall calibre-common it works. I even tested with different combinations in the configuration values.

The tags functionality and downloading of images works as well. Great job.

So conclusion.
Without Python2 it's impossible to run this recipe.
It would be great if you could guarantee python3 support, without python2.
I dont want to have python2 because of calibre on my system, just to run this recipe.

Thanks in advance

[mmagnus]

maybe this ?

#23