wordbreaker breaks on long line

[GoogleCodeExporter]

wordbreaker breaks on long line 
and it is also generally not possible to use dicts bigger than 512 chars.

What steps will reproduce the problem?
1. use a line longer than 512 chars in the dict file (aka wordbreaker-dict.txt)
2. if chat at 511 is not a "space", one gets "bad freqdict line: zero freq"

What is the expected output? 
- program runs
What do you see instead?
- program breaks with "bad freqdict line: zero freq"

What version of the product are you using? 
- development
On what operating system?
- linux
Please provide any additional information below.
- program should support also multi-lines dict files.
- program should read line by line till end of file.

Original issue reported on code.google.com by aab...@gmail.com on 9 Jun 2013 at 10:56