Wallabag plugin #4271

y-muller · 2018-10-14T19:51:39Z

A plugin to download articles from a Wallabag server, an open source read-it-later service (https://wallabag.org).

Issues:

(solved) The settings menu changes the menu item text for the download directory and the tag filter. It makes it easy to see the current configuration. But I have not found how to refresh the value after a change. I could remove the feature but I'd rather have a hint on how to fix it.
To skip files that already exist locally, I want to check the dates of the local and remote versions. The only date parser is in the newsdownloader plugin. I have added an ugly hack to use it if available. Thoughts?

Archiving on server non-functional. Missing screen for local settings.

Setting archived status on server doesn't work. Might be a server issue. Switch to branch 'wallabag-archiving' to work on this feature.

Password is obscured. Ugly hack to get parse the time of article.

poire-z · 2018-10-14T20:30:56Z

About your 1st issue I have not found how to refresh the value after a change :

the callback gets provided the touchmenu_instance as its first argument, so you can do something with it inside the callback depending on what happened (most often: close it, or update/refresh it). See some examples in #4189, eg: https://github.com/koreader/koreader/pull/4189/files#diff-01418c46265dd5757e5be1103a3a224e).
In your case, you want to pass it to Wallabag:setDownloadDirectory(touchmenu_instance), which can call just after self:saveSettings() : touchmenu_instance:updateItems()

y-muller · 2018-10-14T20:33:26Z

Can I run the CI checks on my own branch?

Thanks to poire-z.

y-muller · 2018-10-14T22:23:39Z

the callback gets provided the touchmenu_instance as its first argument, so you can do something with it

Thanks, that worked nicely.

Frenzie · 2018-10-15T05:39:47Z

You can enable CircleCI on your own repo, yes. Let me know if you need a pointer!

Frenzie

Nice work!

Frenzie · 2018-10-15T05:41:39Z

plugins/wallabag.koplugin/main.lua

+local _ = require("gettext")
+local T = FFIUtil.template
+local Screen = require("device").screen
+local DataStorage = require("datastorage")


Please note the order in our coding standards.

Frenzie · 2018-10-15T05:42:30Z

plugins/wallabag.koplugin/main.lua

+                        end,
+                    },
+                    {
+                        text = _("Remotely delete Finished articles"),


Capitalization.

plugins/wallabag.koplugin/main.lua

Frenzie · 2018-10-15T05:44:33Z

plugins/wallabag.koplugin/main.lua

+                        keep_menu_open = true,
+                        callback = function()
+                            UIManager:show(InfoMessage:new{
+                                text = _('Download directory: use a directory that is exclusively used by the Wallabag plugin. ' ..


Please use a multiline comment. Our translation string extraction regex will fail on this.

Is a max of 79 chars OK or should I split them further?

Frenzie · 2018-10-15T05:45:20Z

plugins/wallabag.koplugin/main.lua

+                keep_menu_open = true,
+                callback = function()
+                    UIManager:show(InfoMessage:new{
+                        text = T(_('Wallabag is an open source Read-it-later service. This plugin synchronises with a Wallabag server.\n\n' ..


Frenzie · 2018-10-15T05:46:18Z

plugins/wallabag.koplugin/main.lua

+
+    -- Check if the configuration is complete
+    local function isempty(s)
+        return s == nil or s == ''


Nitpick, but could you use double quotes?

Frenzie · 2018-10-15T05:47:49Z

plugins/wallabag.koplugin/main.lua

+
+function Wallabag:getBearerToken()
+    -- workaround for dateparser, only once
+    -- the parser is in newsdownloader.koplugin, check if it is available


Should it be moved?

I moved it to init().

The date parser is really useful. Ideally it would be moved to a place where all plugins can access it.

Frenzie · 2018-10-15T05:49:03Z

plugins/wallabag.koplugin/main.lua

+end
+
+function Wallabag:editServerSettings()
+    local text_info = "Enter the details of your Wallabag server and account.\n"..


Shortened strings a bit.

It's not about length but concat won't work right.

Use either just a single line with \n or [[multiline]]

It's not really a defect in the regex imo (or I would've changed it) because [[]] is much nicer anyway.

I get it now. I have changed it to multiline.

Frenzie · 2018-10-15T12:02:29Z

plugins/wallabag.koplugin/main.lua

+                    end
+                    local num_deleted = self:processLocalFiles( "manual" )
+                    UIManager:show(InfoMessage:new{
+                        text = T(_('Articles processed.\nDeleted: %1'), num_deleted)


Could you also make these double quotes for consistency?

Frenzie · 2018-10-15T12:05:01Z

plugins/wallabag.koplugin/main.lua

+                            else
+                                path = filemanagerutil.abbreviate(self.directory)
+                            end
+                            return _("Set download directory") .. " (" .. path .. ")"


Even for stuff like this it's generally preferable to use T(). The thing is, you don't really know if some localization might like to make it (path) Set download directory or Download directory (path) set or many other possibilities. :-)

Frenzie · 2018-10-15T12:06:19Z

plugins/wallabag.koplugin/main.lua

+                keep_menu_open = true,
+                callback = function()
+                    UIManager:show(InfoMessage:new{
+                        text = T(_([[Wallabag is an open source Read-it-later service. This plugin synchronises with a Wallabag server.


read-it-later should be lowercase

Frenzie · 2018-10-15T12:06:30Z

plugins/wallabag.koplugin/main.lua

+
+    if isempty(self.server_url) or isempty(self.username) or isempty(self.password)  or isempty(self.client_id) or isempty(self.client_secret) or isempty(self.directory) then
+        UIManager:show(InfoMessage:new{
+            text = _('Please configure the server and local settings.')


Frenzie · 2018-10-15T12:06:38Z

plugins/wallabag.koplugin/main.lua

+    logger.dbg("mode:", dir_mode)
+    if dir_mode ~= "directory" then
+         UIManager:show(InfoMessage:new{
+            text = _('The download directory is not valid.\nPlease configure it in the settings.')


Frenzie · 2018-10-15T12:06:49Z

plugins/wallabag.koplugin/main.lua

+        })
+        return false
+    end
+    if string.sub( self.directory, -1 ) ~= '/' then


more quotes ;-)

Frenzie · 2018-10-15T12:07:39Z

plugins/wallabag.koplugin/main.lua

+    end
+
+    local login_url = "/oauth/v2/token"
+    local body = "{ \"grant_type\": \"password\", \"client_id\": \"" .. self.client_id .. "\", \"client_secret\": \"" .. self.client_secret .. "\", \"username\": \"" .. self.username .. "\", \"password\": \"" .. self.password .. "\"}"


Yet perhaps here there's a good reason to deviate and use single quotes? ;-)

PS string.format could potentially be more legible.

Good shout. A string.format and a [[multiline]] made it a lot neater.

Frenzie · 2018-10-15T12:08:57Z

plugins/wallabag.koplugin/main.lua

+        ["Accept"] = "application/json, */*",
+        ["Content-Length"] = tostring(#body),
+        }
+    local result = self:callAPI( 'POST', login_url, headers, body, "" )


I have to wonder actually, is there a specific rationale behind where you used single and where double quotes?

I'm afraid there was no specific logic in my use of quotes.
I starting by grabbing a few bits of interesting code in other plugins and after a while the inconsistencies spread a bit.

All changed to double quotes and a few multilines.

Frenzie · 2018-10-15T12:09:12Z

plugins/wallabag.koplugin/main.lua

+
+function Wallabag:getArticleList()
+    local articles_url = "/api/entries.json?archive=0&tags=" .. self.filter_tag
+    return self:callAPI( 'GET', articles_url, nil, "", "" )


Frenzie · 2018-10-15T12:09:47Z

plugins/wallabag.koplugin/main.lua

+function Wallabag:callAPI( method, apiurl, headers, body, filepath )
+    local request, sink = {}, {}
+    local parsed = url.parse(apiurl)
+    request['url'] = self.server_url .. apiurl


more quotes

Frenzie · 2018-10-15T12:10:03Z

plugins/wallabag.koplugin/main.lua

+    logger.dbg("method  ", method)
+
+    http.TIMEOUT, https.TIMEOUT = 10, 10
+    local httpRequest = parsed.scheme == 'http' and http.request or https.request


Frenzie

These nits should be the last. Earlier I left a couple of quick comments on my phone for code that "smells" at a glance; these were nitpicks on my laptop. ;-)

Frenzie · 2018-10-15T12:12:16Z

plugins/wallabag.koplugin/main.lua

+    logger.dbg("deleting article ", path )
+    local id = self:getArticleID( path )
+    if id then
+        self:callAPI( 'DELETE', "/api/entries/" .. id .. ".json", nil, "", "" )


Frenzie · 2018-10-15T12:14:19Z

plugins/wallabag.koplugin/main.lua

+    local text_info = [[Enter the details of your Wallabag server and account.
+
+Client ID and client secret are long strings so you might prefer to save the empty settings and edit the config file directly:
+.adds/koreader/settings/wallabag.lua


This is Kobo-specific (and even then mainly for KSM), plus I suspect you typod settings? You should populate this with DataStorage:getSettingsDir().

koreader/datastorage.lua

Line 36 in fa0117b

function DataStorage:getSettingsDir()

Frenzie · 2018-10-15T12:19:16Z

plugins/wallabag.koplugin/main.lua

+                            else
+                                filter = self.filter_tag
+                            end
+                            return _("Filter articles by tag") .. " (" .. filter .. ")"


Copying my other comment.

Even for stuff like this it's generally preferable to use T().

https://github.com/koreader/koreader/pull/4271/files#r225139189

Double quotes. T(). Multilines.

Frenzie · 2018-10-15T16:27:43Z

Alright, I'll let it sit a couple more days so feel free to ping me then if I haven't looked on GH yet.

@poire-z or just merge it if you think it's ready. ;-)

poire-z · 2018-10-16T18:48:25Z

Looking fine and clean, so merging.

poire-z · 2018-10-17T20:36:31Z

There might be a little something if one has not yet configured this plugin.
Not yet tried on my kobo, but on the emulator:

10/17/18-22:30:49 INFO  FM loaded plugin terminal at plugins/terminal.koplugin
10/17/18-22:30:49 INFO  FM loaded plugin texteditor at plugins/texteditor.koplugin
10/17/18-22:30:49 ERROR Failed to initialize wallabag plugin:  plugins/wallabag.koplugin/main.lua:44: attempt to index field 'wallabag' (a nil value)

The menu is still registered and usable thus.

koreader/plugins/wallabag.koplugin/main.lua

Lines 42 to 45 in a3a17db

    
           self.ui.menu:registerToMainMenu(self) 
        
           self.wb_settings = self.readSettings() 
        
           self.server_url = self.wb_settings.data.wallabag.server_url 
        
           self.client_id = self.wb_settings.data.wallabag.client_id

Looks like when there is no settings yet, there is no self.wb_settings.data.wallabag table.
Might just need to be set to ...wallabag={} in Wallabag:readSettings() if it's not there.

y-muller · 2018-10-18T17:23:03Z

There might be a little something if one has not yet configured this plugin.

Yes, I've reproduced it after deleting the config file. PR to fix it coming up.
Sorry about that.

bobberb · 2018-11-17T06:44:57Z

The wallabag plugin provides ALMOST 1:1 parity to the pocket integration on the Kobo. This is amazing. Currently, the functionality is focused on server-side deletion post-read.

To accommodate maybe more use cases, maybe, there should be an option that only favorited/unread articled be allowed. Triggers activate when the un-stared article is finished to make the "mark as read" api call. On next sync the article will be removed from the device but kept on the server. At that point, the device could also accept permanently retained articles via a "device" tag.

y-muller · 2018-11-17T10:04:34Z

Thanks @bobberb. I have a couple of bugs to fix on the plugin first. Then I'll have a look at your suggestion.

Marking a document as "finished" is important for Wallabag (see, among others, koreader#4737, koreader#4271). Also it's an obvious missing action. Default to left, up. That keeps with the fullscreen dialog gestures generally go up. Also it was still unassigned.

Marking a document as "finished" is important for Wallabag (see, among others, #4737, #4271). Also it's an obvious missing action. Default to left, up. That keeps with the fullscreen dialog gestures generally go up. Also it was still unassigned. * Remove separator on last item, see #4737 (review)

Marking a document as "finished" is important for Wallabag (see, among others, koreader#4737, koreader#4271). Also it's an obvious missing action. Default to left, up. That keeps with the fullscreen dialog gestures generally go up. Also it was still unassigned. * Remove separator on last item, see koreader#4737 (review)

Yann Muller and others added 7 commits October 12, 2018 21:17

Wallabag plugin: initial commit.

d2aea20

Archiving on server non-functional. Missing screen for local settings.

Fix for VFAT file system. Synchronisation of remote deletions.

11cf038

Much improved settings. A few fixes.

0c4b52c

Removed archiving support.

5f6b5cc

Setting archived status on server doesn't work. Might be a server issue. Switch to branch 'wallabag-archiving' to work on this feature.

Refresh filemanager after downloads

a2ff135

Password is obscured. Ugly hack to get parse the time of article.

Moved menu entry down.

3485598

Fixed some of the CI errors.

72976c9

Update the menu values after a change.

41d0af1

Thanks to poire-z.

Simplified menu update.

7aee7f2

Frenzie requested changes Oct 15, 2018

View reviewed changes

y-muller added 2 commits October 15, 2018 09:06

Fixes from PR comments.

dc6a5c9

Switched long strings to multiline [[]]

afcba1a

Frenzie added the enhancement label Oct 15, 2018

Frenzie reviewed Oct 15, 2018

View reviewed changes

plugins/wallabag.koplugin/main.lua Outdated

})

return false

end

if string.sub( self.directory, -1 ) ~= '/' then

Copy link

Member

Frenzie Oct 15, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

more quotes ;-)

Frenzie reviewed Oct 15, 2018

View reviewed changes

Review comments.

b19fc7d

Double quotes. T(). Multilines.

poire-z merged commit 846f8bf into koreader:master Oct 16, 2018

y-muller deleted the wallabag branch October 17, 2018 08:43

Frenzie mentioned this pull request Mar 6, 2019

[UX] Gesture manager: add action - book status #4746

Merged

Wallabag plugin #4271

Wallabag plugin #4271

Conversation

y-muller commented Oct 14, 2018 • edited Loading

poire-z commented Oct 14, 2018

y-muller commented Oct 14, 2018

y-muller commented Oct 14, 2018 • edited Loading

Frenzie commented Oct 15, 2018

Frenzie left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

y-muller Oct 15, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Frenzie Oct 15, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Frenzie left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Frenzie commented Oct 15, 2018

poire-z commented Oct 16, 2018

poire-z commented Oct 17, 2018

y-muller commented Oct 18, 2018

bobberb commented Nov 17, 2018

y-muller commented Nov 17, 2018

y-muller commented Oct 14, 2018 •

edited

Loading

y-muller commented Oct 14, 2018 •

edited

Loading

y-muller Oct 15, 2018 •

edited

Loading

Frenzie Oct 15, 2018 •

edited

Loading