Release 0.6.1

This release contains the following changes:

• optimize the n-gram model, change grams from [2,3] to [2,3,4]
• append placeholder ^ for short words
• fix a bug in the n-gram model where the ^$ placeholder notes are not actually added to the word

This release contains the following resources:

• ngram.json.zip: updated n-gram model weights

The rest of the resources are unchanged from 0.6.0.

Release 0.6.0

This release adds the following resources:

• ngram.json.zip: the zipped JSON file containing weights and threshold for a bi-gram and a tri-gram model

The resources unchanged from tag 0.5.0 are not included:

• accent.npz: exported npz file directly used in the accent predictor, fp16
• accent.safetensors: exported safetensors file from accent predictor for integration in other languages, fp16
• accent.pth: raw state_dict file from pretrained accent predictor in PyTorch, fp32, 20 epochs
• katakana_dict.jsonl: a JSON line file contains the extracted English to Katakana dictionary
• model-{c2k/p2k}.npz: npz files directly used in production library, fp16
• model-{c2k/p2k}.safetensors: safetensors files for integration in other languages, fp16
• model-{c2k/p2k}-e10.pth: raw state_dict files from pretrained models in PyTorch, fp32, 10 epochs

Check 0.5.0 for these resources.

Release 0.5.1

Adds an accent predictor.

This release contains the resources

• accent.npz: exported npz file directly used in the accent predictor, fp16
• accent.safetensors: exported safetensors file from accent predictor for integration in other languages, fp16
• accent.pth: raw state_dict file from pretrained accent predictor in PyTorch, fp32, 20 epochs

Resources that doesn't change from 0.4.0 is not included:

• katakana_dict.jsonl: a JSON line file contains the extracted English to Katakana dictionary
• model-{c2k/p2k}.npz: npz files directly used in production library, fp16
• model-{c2k/p2k}.safetensors: safetensors files for integration in other languages, fp16
• model-{c2k/p2k}-e10.pth: raw state_dict files from pretrained models in PyTorch, fp32, 10 epochs

Check 0.4.0 for these resources.

Release 0.4.1

What's new:

• Packs metadata (in_table, out_table, sos/eos index) with the exported model weights

Breaking Change

• Improve the API to non-stateful calls, see usage for more information

This release contains following resources:

• model-{c2k/p2k}.npz: npz files directly used in production library, fp16
• model-{c2k/p2k}.safetensors: safetensors files for integration in other languages, fp16

Resources that doesn't change from 0.3.0 is not included:

• katakana_dict.jsonl: a JSON line file contains the extracted English to Katakana dictionary
• model-{c2k/p2k}-e10.pth: raw state_dict files from pretrained models in PyTorch, fp32, 10 epochs

Check 0.3.0 for these resources.

Be acknowledged that the dictionary should follow the Wikimedia's license and the JMdict / EDICT's Copyright license.

Release 0.3.0

What's new:

• Removed unused entries in C2K (upper case ASCII, they were there for a mistake).
  • It changes the embedding dim, I treat this as an API change and bump the minor version.
• Use a slightly larger training split.
• Add ' into input symbols for C2K and P2K, this boosted the bleu score a little bit.

This release contains following resources:

• model-{c2k/p2k}-e10.pth: raw state_dict files from pretrained models in PyTorch, fp32, 10 epochs
• model-{c2k/p2k}.npz: npz files directly used in production library, fp16
• model-{c2k/p2k}.safetensors: safetensors files for integration in other languages, fp16

Resources didn't change from 0.2.0 is not included:

• katakana_dict.jsonl: a JSON line file contains the extracted English to Katakana dictionary

Check 0.2.0 for these resources.

Please acknowledge that the dictionary should follow the Wikimedia's license and the JMdict / EDICT's Copyright license.

I'm not a lawyer, how to treat the model weights derived from the dictionary is up to you.

Release 0.2.0

This release contains following resources:

• katakana_dict.jsonl: a JSON line file contains the extracted English to Katakana dictionary
• model-{c2k/p2k}-e10.pth: raw state_dict files from pretrained models in PyTorch, fp32
• model-{c2k/p2k}.npz: npz files directly used in production library, fp16
• model-{c2k/p2k}.safetensors: safetensors files for integration in other languages, fp16

Please acknowledge that all the resources should follow the Wikimedia's license and the JMdict / EDICT's Copyright license, which, in short, both need proper attribution in your work.

Initial Release 0.1.1

This release includes:

• katakana_dict.jsonl: The parsed English-Katakana dictionary, with 32K entries
• model-p2k.npz: The model weights in fp16 npz file, for phoneme (cmudict) to Katakana usage.
• model-c2k.npz: The same format as above, for English characters to Katakana usage.

Note: both models support white space " " as input, but don't output white spaces.