BIP39: Added ukrainian wordlist

[Bohdat]

No description provided.

[luke-jr]

@slush0 @prusnak @voisine @ebfull

[voisine]

this needs to be NFKD normalized, which you can do with the following perl script:

#!/usr/bin/perl

use Unicode::Normalize;
use strict;
use warnings;
use open qw(:std :utf8);

while (<>) {
    print NFKD("$_");
}

[voisine]

Looks good to me, but I'd like a second ukranian speaker to go over the list and verify it meets the word list criteria before ACKing

[greenaddress]

we reviewed the words (ukranian speaker) and they look OK - however the list doesn't seem sorted (run sort on it, export LANG=C first if you don't have it set).

If sorted it allows faster processing (binary search) and we think it is worthwhile doing it.

[zerko]

Doesn't look like words are identifiable by first four letters.

[slush0]

Script for validating all BIP39 defined rules (like uniqueness of first four letters) is here:
https://github.com/trezor/python-mnemonic/blob/master/test_mnemonic.py

Maybe it will need fixes for UTF-8 (eventually slight rewrite for python3 which handle UTF much better), but passing such tests is needed for adding into BIP.

[slush0]

Okay, I run test_mnemonic.py (with Python3 - with no problems) and it gave me such list of duplicates: http://pastebin.com/ztBqDT9q

There were some other minor errors, but this need some work.

[nym-zone]

At nym-zone/easyseed@08a05b4, I have created a bugfixed ukrainian.txt which is NFKD-normalized and binary-sorted, and fixes one technical bug.

The ukrainian.txt from Bohdat/bips@152fc59 contains a trailing space (0x20) then tab (0x09, '\t') after the word at original index 1393 (1-based line number 1394), before the newline '\n'. The problem was first identified by failure of easyseed’s extensive internal self-tests, followed by examination with cmp(1) and hex dumps to diagnose the difference between the wordlist in my source tree, and the wordlist printed on stdout by easyseed -W -P -l uk.

The following commands pinpoint the problem:

$ grep -E '[[:space:]]$' ukrainian.txt | hd
00000000  d0 bf d1 96 d1 81 d0 bd  d1 8f 20 09 0a           |.......... ..|
0000000d
$ echo "\"`grep -En '[[:space:]]$' ukrainian.txt`\""
"1394:пісня 	"

(@dabura667, perhaps you may want to add that to your punch-list of technical checks.)

It is fixed with the following command:

$ sed -E -e 's/[[:space:]]+$//' < ukrainian.txt > ukfix1/uk_fixed0.txt

After verification that this command made no other changes, the list is normalized and sorted:

$ ls -l ukrainian.txt ukfix1/uk_fixed0.txt
-rw-r--r-- 1 user user 24550 Jan  7 21:26 ukfix1/uk_fixed0.txt
-rw-r--r-- 1 user user 24552 Jan  7 20:31 ukrainian.txt
$ diff -u3 ukrainian.txt ukfix1/uk_fixed0.txt
[...showing only the desired line changed...]
$ uconv -f utf-8 -t utf-8 -x '::nfkd;' < uk_fixed0.txt | \
	LC_ALL=C LANG=C sort -s > uk_fixed1.txt
$ mv -i uk_fixed1.txt ../../easyseed/wordlist/ukrainian.txt
mv: overwrite '../../easyseed/wordlist/ukrainian.txt'? y

SHA-256 hash for the resulting ukrainian.txt:

612ee29e1fa13dc38c9e1b31c7ef980db8f3c8dd30f1c9377170d1b10e895dc9

[kittyandrew]

Hello, I've almost started working on my own list for this, and found this pr. Can someone tell me what exactly here needs to be fixed/updated/reviewed?

Follow up question: are there any preferences for nouns-verbs-adjectives? There are (at least) few special words that translate to "and", "or", "there" etc.

Edit: In addition, there are many closely related words and different forms of the same word - working on those.

Edit 2: probably will have to create new PR in the end, because current contributor is inactive.

[luke-jr]

For now, the author(s) of BIP 39 have decided not to accept any further word lists into BIP 39 itself, and encourage adding new ones to the WLIPs repo here: https://github.com/p2w34/wlips