Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

way to remove string resource when it changes to empty. #2276

Closed
tateisu opened this issue Sep 27, 2018 · 3 comments
Closed

way to remove string resource when it changes to empty. #2276

tateisu opened this issue Sep 27, 2018 · 3 comments
Assignees
Labels
enhancement Adding or requesting a new feature. translate-toolkit Issues which need to be fixed in the translate-toolkit undecided These features might not be implemented. Can be prioritized by sponsorship.
Milestone

Comments

@tateisu
Copy link

tateisu commented Sep 27, 2018

Is your feature request related to a problem? Please describe.

I have experienced it in my repository https://hosted.weblate.org/projects/subway-tooter/ .

a contributor makes empty some strings that had same content with default language.
seems he expects "fallback to default language".

then weblate makes empty string resources for those.
those will be handled as empty string resource on android resource system,
NOT fallback to default language.

Describe the solution you'd like

provide a way to remove a string resource in specified language.
it will not be encoded in strings.xml.

Describe alternatives you've considered

force remove a string resource when empty string is set.

Additional context
Add any other context or screenshots about the feature request here.

@nijel nijel added enhancement Adding or requesting a new feature. undecided These features might not be implemented. Can be prioritized by sponsorship. translate-toolkit Issues which need to be fixed in the translate-toolkit labels Sep 28, 2018
@nijel
Copy link
Member

nijel commented Sep 28, 2018

It probably makes sense to avoid storing empty strings on monolingual translations, this however needs to be implemented in the translate-toolkit level first - there is no API currently to remove unit from the file (see also #1974).

Anyway my recommendation is that most translations will not be aware of this, so it's best to simply copy the source string and it's done. If you don't want to translate string at all, just remove it from translatable strings.

@eighthave
Copy link
Contributor

eighthave commented May 3, 2019

For F-Droid, we have a script that we run on release branches that removes all incomplete translations. It actually deletes the incomplete translations, so you probably don't want to run this on master.

(oops sorry for the multiple edits, its too early...)

https://gitlab.com/fdroid/fdroidclient/blob/master/tools/trim-incomplete-translations-for-release.py

# WARNING!  THIS DELETES TRANSLATIONS!
#
# The incomplete translations should be kept by rebasing the weblate
# remote on top of this commit, once its complete.

import csv
import git
import os
import requests
import sys


projectbasedir = os.path.dirname(os.path.dirname(__file__))
print(projectbasedir)

repo = git.Repo(projectbasedir)

msg = 'removing all translations less than 70% complete\n\n'

url = 'https://hosted.weblate.org/exports/stats/f-droid/f-droid/?format=csv'
r = requests.get(url)
stats = csv.reader(r.iter_lines(decode_unicode=True), delimiter=',')
next(stats)  # skip CSV header
for row in stats:
    if len(row) > 4:
        if float(row[4]) > 70.0:
            continue
        locale = row[1]
        if '_' in locale:
            codes = locale.split('_')
            if codes[1] == 'Hans':
                codes[1] = 'CN'
            elif codes[1] == 'Hant':
                codes[1] = 'TW'
            locale = codes[0] + '-r' + codes[1]
        translation_file = 'app/src/main/res/values-' + locale + '/strings.xml'
        percent = str(int(float(row[4]))) + '%'
        print('Removing incomplete file: (' + percent + ')\t',
              translation_file)
        delfile = os.path.join(projectbasedir, translation_file)
        if os.path.exists(delfile):
            os.remove(delfile)
            repo.index.remove([translation_file, ])
        if len(percent) == 2:
            msg += ' '
        msg += percent + '  ' + row[1] + '  ' + row[0] + '\n'

found = False
for remote in repo.remotes:
    if remote.name == 'weblate':
        remote.fetch()
        found = True

if not found:
    print('ERROR: there must be a weblate remote to preserve incomplete translations!')
    sys.exit(1)

repo.index.commit(msg)

@eighthave
Copy link
Contributor

Actually, I forgot that one is pretty raw. We have a script for the website that is safer, it only cherry picks completed translations. Using this script requires using the "squash/rebase" addon:

https://gitlab.com/fdroid/fdroid-website/blob/master/tools/pick-complete-translations.py

import git
import json
import os
import re
import requests
import sys
import yaml


def get_paths_tuple(locale):
    return (
        '_data/%s/strings.json' % locale,
        '_data/%s/tutorials.json' % locale,
        'po/_docs.%s.po' % locale,
        'po/_pages.%s.po' % locale,
        'po/_posts.%s.po' % locale,
    )

projectbasedir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

repo = git.Repo(projectbasedir)
weblate = repo.remotes.weblate
weblate.fetch()
upstream = repo.remotes.upstream
upstream.fetch()

url = 'https://hosted.weblate.org/exports/stats/f-droid/website/?format=json'
r = requests.get(url)
r.raise_for_status()
strings = r.json()

url = 'https://hosted.weblate.org/exports/stats/f-droid/website-pages/?format=json'
r = requests.get(url)
r.raise_for_status()
pages = r.json()

with open(os.path.join(projectbasedir, '_config.yml')) as fp:
    site_languages = yaml.load(fp)['languages']

strings_entries = dict()
pages_entries = dict()

merge_locales = []
for entry in strings:
    strings_entries[entry['code']] = entry
for entry in pages:
    pages_entries[entry['code']] = entry

for locale in site_languages:
    s = strings_entries.get(locale)
    p = pages_entries.get(locale)
    if s is not None and s['translated_percent'] == 100 and s['failing'] == 0 \
       and p is not None and p['translated_percent'] > 98 and p['failing'] == 0:
        merge_locales.append(locale)

if not merge_locales:
    sys.exit()

if 'merge_weblate' in repo.heads:
    merge_weblate = repo.heads['merge_weblate']
    repo.create_tag('previous_merge_weblate', ref=merge_weblate,
                    message=('Automatically created by %s' % __file__))
else:
    merge_weblate = repo.create_head('merge_weblate')
merge_weblate.set_commit(upstream.refs.master)
merge_weblate.checkout()

email_pattern = re.compile(r'by (.*?) <(.*)>$')

for locale in sorted(merge_locales):
    commits = list(repo.iter_commits(
        str(weblate.refs.master) + '...' + str(upstream.refs.master),
        paths=get_paths_tuple(locale), max_count=10))
    for commit in reversed(commits):
        repo.git.cherry_pick(str(commit))
        m = email_pattern.search(commit.summary)
        if m:
            email = m.group(1) + ' <' + m.group(2) + '>'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Adding or requesting a new feature. translate-toolkit Issues which need to be fixed in the translate-toolkit undecided These features might not be implemented. Can be prioritized by sponsorship.
Projects
None yet
Development

No branches or pull requests

3 participants