Porting this awesome lib for another language - guidelines

[BERA]

Hi

First off all I want to congrats everyone involved in the development of this project.

I'm interested to improve the usage of this library for Portuguese language, so to achieve it I need to research for common words in and most popular password words in this language to build a more accurate bad password list, I guess.

There is some information describing the process for change the code to provide the dictionary for a set of words in another language, or maybe some simple approach to use this lib with bad password list and not permitted password in another language too?

Thanks!

[lowe]

Hi Bera,

This is indeed very doable!

Question for you -- it isn't enough to have a dictionary for another
language. It needs to be a ranked dictionary, where you have some sense of
common (a, the, banana) vs uncommon (adolescent, optometrist). Did you come
across such a list for Portuguese? It'd be fun to build your own too with
e.g. nltk and a nice corpus.

Recommended steps:

1. install zxcvbn dependencies: coffeescript, java, python, and the
   simplejson python module
2. clone zxcvbn, confirm these steps work for you:
   cd zxcvbn/scripts
   python build_frequency_lists.py
   cd ..
3. adapt build_frequency_lists.py to add your Portuguese lists and
   (optionally) remove the English lists. I recommend doing this by adding
   your datasource to zxcvbn/data and making as minor a change as possible to
   build_frequency_lists.py to read it in.

Hope that helps, let me know if you have any other questions.
Dan

On 13 May 2014 06:49, Bera notifications@github.com wrote:

Hi

First off all I want to congrats everyone involved in the development of
this project.

I'm interested to improve the usage of this library for Portuguese
language, so to achieve it I need to research for common words in and most
popular password words in this language to build a more accurate bad
password list, I guess.

There is some information describing the process for change the code to
provide the dictionary for a set of words in another language, or maybe
some simple approach to use this lib with bad password list and not
permitted password in another language too?

Thanks!

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43
.

[pyramids]

Google n-grams might be an acceptable source for ranked dictionaries. Freely available at
http://commondatastorage.googleapis.com/books/syntactic-ngrams/index.html

I suppose tweet word frequency lists could be an even better estimate for casually spelled and keyboard-based word usage, but I have no source for those.

EDIT:
I gave a bad (English-only) link to Google's n-gram data. This one covers more languages, but still does not include Portuguese: http://storage.googleapis.com/books/ngrams/books/datasetsv2.html

[lowe]

Good idea!

On 13 May 2014 14:44, Björn Stein notifications@github.com wrote:

Google n-grams might be an acceptable source for ranked dictionaries.
Freely available at
http://commondatastorage.googleapis.com/books/syntactic-ngrams/index.html

I suppose tweet word frequency lists could be an even better estimate for
casually spelled and keyboard-based word usage, but I have no source for
those.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/43#issuecomment-43017223
.

[BERA]

Thank you guys! I'll try and let you know soon.

[ebeeson]

@BERA did you ever have any success with this?

[BERA]

Hi Erik

Unfortunately in this year I gave up of it. But I'm planning to add this in
my todo list in the next year.

This will be a great experience for sure. Maybe I can port this for golang
and make an API for IT.

Thanks for your contact and I'm apologize for let this abandon issue for
now.

Em qui, 5 de nov de 2015 às 17:40, Erik Beeson notifications@github.com
escreveu:

@BERA https://github.com/Bera did you ever have any success with this?

—
Reply to this email directly or view it on GitHub
#43 (comment).

[Esekil]

Hello, I would like to add an Italian dictionary to this library. As a first step I added to the file data my dictionary and I modified the file built_frequency_list.py:
DICTIONARIES = dict
(
us_tv_and_film = 30000,
english_wikipedia = 30000,
passwords = 30000,
surnames = 10000,
male_names = None,
female_names = None,
italian_dictionary = None, )
adding the least line before ")" , unfortunately, I could not to compile file built_frequency_list.py. Would you please help me to figure out how to do it. Thank you for your time.

[pepve]

I forked this repository and made some adjustments for Dutch. I added first and last names, and I added words from the Dutch Wikipedia using the same method as for English.

Repository here: https://github.com/pepve/zxcvbn-nl

Relevant commit here: pepve/zxcvbn-nl@30fad91

[JoSSte]

maybe http://letterfrequency.org/letter-frequency-by-language/ could assist in porting the library to other languages...

[flaviogrossi]

To anyone interested, I'm working on adding italian words and names to zxcvbn here, based on wikipedia entries and common italian names.

[RawanAyoub]

To help anyone that sat like me with no pyhton experience at alll:
Add your files to the data folder,
Change build_frequency_lists.py so it includes your file name as a dict,
run python build_frequency_lists.py ../data ../src/frequency_lists.coffee,
run npm install

That created a new .js file which contain the new dictionaries!

Good luck to anyone having this issue

[RawanAyoub]

And you need pyhon 2 because some things are deprecated in python 3. For example iterItems() are now items().