Skip to content

v.0.1.0

Latest
Latest

Choose a tag to compare

View all tags
@armish armish released this 17 Sep 13:26
v0.1.0
70a73f2

Release v0.1.0 - Turkish Diacritics Restoration with Advanced Architecture

🎉 Major Release Highlights

This release introduces significant architectural improvements that reduce model complexity by 50% while improving accuracy on capital letters and sentence beginnings. The system now features Turkish-specific linguistic handling, CUDA compatibility, and flexible data preparation tools.

✨ Key Features

🧠 Advanced Neural Architecture

  • BiLSTM + Multi-Head Self-Attention: State-of-the-art sequence modeling for Turkish text
  • Case Normalization Pipeline: Process text in lowercase with deterministic case restoration
  • Binary Classification: 6 specialized classifiers for Turkish diacritic pairs (c/ç, g/ğ, i/ı, o/ö, s/ş, u/ü)
  • Context-Aware Processing: 96-character context windows for accurate diacritic restoration
  • Model complexity: 6 classification heads (lowercase only with case pattern restoration). Faster training, better convergence, improved capital letter accuracy.

🇹🇷 Turkish-Specific Linguistic Handling

  • Proper i/İ and ı/I Mappings: Critical for Turkish orthography
    • i → İ (dotted lowercase to dotted uppercase)
    • ı → I (dotless lowercase to dotless uppercase)
  • Vowel Harmony Support: Captures Turkish agglutination patterns
  • Conservative Predictions: Model learns when NOT to add diacritics

🚀 Multi-Platform Support

  • CUDA GPUs: Full compatibility with NVIDIA hardware (A100, V100, RTX/GTX series)
  • Apple Silicon MPS: Optimized for M1/M2/M3 processors
  • CPU Fallback: Automatic detection with performance warnings
  • Platform-Specific Configs: --cuda-config, --mps-config, --cpu-config presets

Loss Function Design

  • Binary CrossEntropy Loss: Equal penalties for false positives and negatives
  • Frequency Weighting: Rare characters get appropriate attention
  • Balanced Training: Model learns to preserve non-diacritic characters

📊 Performance Metrics

Model Specifications

20250916_ctx20_hs128_e50-m1_mps-small_model.pth

Trained on M1 MPS via:

nokta-train \
  --data-cache data/combined_cache.pkl \
  --output models/20250916_ctx20_hs128_e50-m1_mps-small_model.pth \
  --context-size 20 \
  --hidden-size 128 \
  --num-lstm-layers 2 \
  --use-attention \
  --epochs 50 \
  --max-train-texts 10000 \
  --max-val-texts 1000 \
  --balanced-sampling \
  --batch-size 32

Performance metrics on test files:

==> aysnrgenc_turkishdeasciifier_test.txt.log <==

Overall character accuracy: 97.79%
Overall word accuracy: 85.62%
Diacritic-specific accuracy: 84.99%

==> llm_random_test.txt.log <==

Overall character accuracy: 97.48%
Overall word accuracy: 83.32%
Diacritic-specific accuracy: 82.31%

==> vikipedi_test.txt.log <==

Overall character accuracy: 98.95%
Overall word accuracy: 93.17%
Diacritic-specific accuracy: 92.28%

20250916_ctx20_hs128_e50-a100-small_model.pth

Trained on NVIDIA A100 via:

nokta-train --data-cache data/combined_cache.pkl --output models/20250916_ctx20_hs128_e50-a100-small_model.pth --context-size 20 --hidden-size 128 --num-lstm-layers 2 --use-attention --epochs 50 --max-train-texts 100000 --max-val-texts 10000 --balanced-sampling --batch-size 32

Performance metrics on test files:

==> aysnrgenc_turkishdeasciifier_test.txt.log <==

Overall character accuracy: 99.91%
Overall word accuracy: 99.42%
Diacritic-specific accuracy: 99.66%

==> llm_random_test.txt.log <==

Overall character accuracy: 99.83%
Overall word accuracy: 98.85%
Diacritic-specific accuracy: 99.02%

==> vikipedi_test.txt.log <==

Overall character accuracy: 99.91%
Overall word accuracy: 99.39%
Diacritic-specific accuracy: 99.56%

20250916_ctx96_hs256_e50-a100-large_model.pth

Trained on NVIDIA A100 via:

nokta-train --data-cache data/combined_cache.pkl --output models/20250916_ctx96_hs256_e50-a100-large_model.pth` --context-size 96 --hidden-size 256 --num-lstm-layers 2 --use-attention --epochs 50 --max-train-texts 100000 --max-val-texts 10000 --balanced-sampling --batch-size 128

Performance metrics on test files:

==> aysnrgenc_turkishdeasciifier_test.txt.log <==

Overall character accuracy: 99.93%
Overall word accuracy: 99.50%
Diacritic-specific accuracy: 99.60%

==> llm_random_test.txt.log <==

Overall character accuracy: 99.87%
Overall word accuracy: 99.11%
Diacritic-specific accuracy: 99.09%

==> vikipedi_test.txt.log <==

Overall character accuracy: 99.93%
Overall word accuracy: 99.50%
Diacritic-specific accuracy: 99.63%

📦 Installation

# From source
git clone https://github.com/armish/nokta-ai.git
cd nokta-ai
git checkout v0.1.0
pip install -e .

🚀 Quick Start

nokta restore --model path/to/model.pth --text "Turkiye"

📝 License

MIT License - See LICENSE file for details

🔗 Links

Assets 5
Loading