Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve and translate Fdroid metadata #8225

Closed
3 tasks done
pazos opened this issue Sep 17, 2021 · 12 comments
Closed
3 tasks done

Improve and translate Fdroid metadata #8225

pazos opened this issue Sep 17, 2021 · 12 comments

Comments

@pazos
Copy link
Member

pazos commented Sep 17, 2021

Each new package at https://f-droid.org/en/packages/org.koreader.launcher.fdroid/ pulls metadata from https://github.com/koreader/koreader/tree/master/metadata/

@Frenzie
Copy link
Member

Frenzie commented Sep 19, 2021

push/pull to/from weblate? need help here :p

None of us know — or rather, you'd have to do some things on https://hosted.weblate.org/create/component/ with the manual in another window.

Do we want that kind of translation activity on the main repo?

@pazos
Copy link
Member Author

pazos commented Jan 12, 2022

Do we want that kind of translation activity on the main repo?

I guess it is ok to pull translations manually once a year :)

@pazos
Copy link
Member Author

pazos commented May 6, 2022

Other way of doing translations (maybe too odd) is to use the regular translations, append the whole fdroid metadata as a single string and write a tool that generates the plain text for each language. So the workflow would be - again, once a year or so -:

cd koreader
tools/update-metadata.lua

In that example the update-metadata.lua will iterate over translations and just write the ones that don't match the english version.

@Frenzie
Copy link
Member

Frenzie commented May 7, 2022

I don't know about odd but it does sound like reinventing the wheel. It seems like a topic they'd cover over on F-Droid somewhere?

@pazos
Copy link
Member Author

pazos commented May 7, 2022

It seems like a topic they'd cover over on F-Droid somewhere?

Actually the whole point about having metadata ourselves was/is to have some degree of control over screenshots and app description. First was #5726 but fdroid server wasn't able to fetch metadata from a submodule, so it was changed to the main repo in 9b6483e

I don't know about odd but it does sound like reinventing the wheel.

A bit but we need to translate only two strings and the whole script is trivial.

I just wrote a test script for the fun of it:

#!/usr/bin/env luajit

require "defaults"
package.path = "frontend/?.lua;" .. package.path

local function isLocaleDir(str)
    for _, v in ipairs({"templates", "README.md", "LICENSE"}) do
        if str == v then return false end
    end
    return true
end

local function testAString(str, path)
    local locales = {}
    local output = io.popen('ls l10n')
    if not output then return end
    for dir in output:lines() do
        if isLocaleDir(dir) then
            locales[#locales + 1] = dir
        end
    end
    output:close()

    print('translation for "' .. str .. '"')
    for __, lang in ipairs(locales) do
        local _ = require("gettext")
        _.changeLang(lang)
        local translation = _(str)
        if str == translation then
            print(lang, "not translated")
        else
            print(lang, '"' .. translation .. '"')
        end
    end
end

testAString("Open with…")

It works but seems to got hit by a bug or something is missing for a specific locale

pazos@bowie~/koreader> ./tools/update-metadata.lua
translation for "Open with…"
af_ZA	not translated
ar	"فتحٌ باستخدام…"
be	not translated
bg_BG	"Отвори с…"
bn	"দিয়ে খুলুন…"
ca	"Obrir amb…"
cs	"Otevřít pomocí…"
de	"Öffne mit…"
el	"Άνοιγμα με…"
eo	"Malfermi per…"
es	"Abriendo con…"
eu	"Ireki honekin..."
fa	not translated
fi	not translated
fr	"Ouvrir avec…"
gl	"Abrir con…"
he	"פתיחה באמצעות…"
hi	not translated
hr	"Otvori pomoću …"
hu	"Megnyitás ezzel…"
id	not translated
ie	not translated
it_IT	"Apri con…"
ja	"開く…"
ka	"გახსნა პროგრამით…"
ko_KR	"다음으로 열기…"
lt_LT	not translated
luajit: frontend/gettext.lua:136: attempt to call a nil value
stack traceback:
	frontend/gettext.lua:136: in function 'getPluralFunc'
	frontend/gettext.lua:243: in function 'changeLang'
	./tools/update-metadata.lua:27: in function 'testAString'
	./tools/update-metadata.lua:37: in main chunk
	[C]: at 0x55731e6602e0

@Frenzie
Copy link
Member

Frenzie commented May 7, 2022

Alright, that looks pretty decent. So we just have a few "ghost" strings as part of the regular flow. Works for me. 👍

Latvian is a language that seems to have been translated a decent amount recently but isn't exposed in the GUI yet. It's likely the plural form can't quite be parsed by the limited parser I wrote. Something along the lines of your script there would actually be a good thing to add to the CI tests. :-)

Frenzie added a commit to Frenzie/koreader that referenced this issue May 7, 2022
Frenzie added a commit that referenced this issue May 7, 2022
@pazos
Copy link
Member Author

pazos commented May 8, 2022

@Frenzie: great. Now it works fine, thanks!

I updated the script to do what's intended to do. A question: do translated strings get picked from tools/update_metadata.lua and made available on weblate automatically? Is some other magic required?.

If the answer to the first question is "yes" then it should work as is

#!/usr/bin/env luajit

-- utility to generate fastlane metadata for each translation
-- usage: ./tools/update_metadata.lua


-- app short description
local template_short = "Ebook reader with support for many formats like PDF, DjVu, EPUB, FB2, CBZ."


-- app full description
local template_full = [[

* portable: runs on embedded devices (Cervantes, Kindle, Kobo, PocketBook), Android and Linux computers. Developers can run a KOReader emulator in Linux and MacOS.

* multi-format documents: supports fixed page formats (PDF, DjVu, CBT, CBZ) and reflowable e-book formats (EPUB, FB2, Mobi, DOC, CHM, TXT). Scanned PDF/DjVu documents can also be reflowed with the built->

* full-featured reading: multi-lingual user interface with a highly customizable reader view and many typesetting options. You can set arbitrary page margins, override line spacing and choose external fo>

* integrated with calibre (search metadata, receive ebooks wirelessly, browse library via OPDS),  Wallabag, Wikipedia, Google Translate and other content providers.

* optimized for e-ink devices: custom UI without animation, with paginated menus, adjustable text contrast, and easy zoom to fit content or page in paged media.

* extensible via plugins

* and much more: look up words with StarDict dictionaries / Wikipedia, add your own online OPDS catalogs and RSS feeds, online over-the-air software updates, an FTP client, an SSH server, …
]]

require "defaults"
package.path = "frontend/?.lua;" .. package.path

local _ = require("gettext")


local metadata = {
    source_short = template_short,
    source_full = template_full,
    translated_short = _(template_short),
    translated_full = _(template_full),
}

local metadata_files = {
    ["short_description.txt"] = metadata.source_short,
    ["full_description.txt"] = metadata.source_full,
}

local function isFile(str)
    local f = io.open(str, "r")
    if f then
        f:close()
        return true
    end
    return false
end

local function writeFile(str, path)
    local f = io.open(path, "w")
    if f then
        f:write(str)
        f:write("\n")
        f:close()
    end
end

local function isLocaleDir(str)
    for _, v in ipairs({"templates", "README.md", "LICENSE"}) do
        if str == v then return false end
    end
    return true
end

local function getLocales()
    local locales = {}
    local output = io.popen('ls l10n')
    if not output then return {} end
    for dir in output:lines() do
        if isLocaleDir(dir) then
            locales[#locales + 1] = dir
        end
    end
    output:close()
    return locales
end

local function updateMetadata(str, path)
    local locales = getLocales()
    local count = { new = 0, updated = 0, not_translated = 0 }
    for __, lang in ipairs(locales) do
        _.changeLang(lang)
        local translation = _(str)
        if str ~= translation then
            local metadata_dir = "metadata/" .. lang
            local metadata_file = metadata_dir .. "/" .. path
            os.execute('mkdir -p ' .. metadata_dir)
            if isFile(metadata_file) then
                count.updated = count.updated + 1
            else
                count.new = count.new + 1
            end
            writeFile(translation, metadata_file)
        else
            count.not_translated = count.not_translated + 1
        end
    end
    print(path, "new", count.new, "updated", count.updated, "not translated", count.not_translated)
end

-- update english metadata first
local en_metadata = "metadata/en-US/"
for file, str in pairs(metadata_files) do
    writeFile(str, en_metadata .. file)
end

-- update translated metadata
for file, str in pairs(metadata_files) do
    updateMetadata(str, file)
end

@Frenzie
Copy link
Member

Frenzie commented May 8, 2022

I updated the script to do what's intended to do. A question: do translated strings get picked from tools/update_metadata.lua and made available on weblate automatically? Is some other magic required?.

Yes, the required magic is that they're surrounded by _(). ^_^ Your sample above is just a random string without that.

@Frenzie
Copy link
Member

Frenzie commented May 8, 2022

Btw, don't bother the translators with stuff like leading newlines. Concatenate those outside of the text to be translated.

I.e.,

"\n\n" .. _([[to be translated]])

@pazos
Copy link
Member Author

pazos commented May 8, 2022

I updated the script to do what's intended to do. A question: do translated strings get picked from tools/update_metadata.lua and made available on weblate automatically? Is some other magic required?.

Yes, the required magic is that they're surrounded by _(). ^_^ Your sample above is just a random string without that.

I used the magic trick in

local metadata = {
    source_short = template_short,
    source_full = template_full,
    translated_short = _(template_short),
    translated_full = _(template_full),
}

😄

@Frenzie
Copy link
Member

Frenzie commented May 9, 2022

Which won't work to pick up the strings.

@pazos
Copy link
Member Author

pazos commented Nov 22, 2022

Done in ee35257

@pazos pazos closed this as completed Nov 22, 2022
@Frenzie Frenzie added this to the 2022.11 milestone Nov 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants