Skip to content

v1.0.1

Choose a tag to compare

@sudoping01 sudoping01 released this 24 Jan 21:38
· 57 commits to main since this release

Bambara Text Normalizer v1.0.1

Release Date: 24th January 2025

License: MIT


What's New

This release adds Measurement Normalization to the TN/ITN toolkit, enabling bidirectional conversion between standard measurement units and Bambara expressions.


New Features

Measurement Normalization (TN/ITN)

from bambara_normalizer import measurement_to_bambara, bambara_to_measurement, normalize

# Weight
measurement_to_bambara(5, "kg")
# → "kilogaramu duuru"

# Length
measurement_to_bambara(100, "m")
# → "mɛtɛrɛ kɛmɛ"

# Volume
measurement_to_bambara(2.5, "L")
# → "litiri fila tomi duuru"

# Area
measurement_to_bambara(3, "ha")
# → "ɛkitari saba"

# Inverse conversion
bambara_to_measurement("kilogaramu duuru")
# → (5, 'kg')

# In text
normalize("A ye 5 kg san", expand_measurements=True)
# → "a ye kilogaramu duuru san"

normalize("So in bɛ 100 m", preset="wer")
# → "so in bɛ mɛtɛrɛ kɛmɛ"

Supported Measurement Units

Category Units Bambara
Weight kg, g, mg, t kilogaramu, garamu, miligaramu, tɔni
Length km, m, cm, mm kilomɛtɛrɛ, mɛtɛrɛ, santimɛtɛrɛ, milimɛtɛrɛ
Volume L, mL litiri, mililitiri
Area ha, m² ɛkitari, mɛtɛrɛ kare

Configuration

from bambara_normalizer import normalize, BambaraNormalizerConfig
# Individual flag
normalize("A ye 5 kg san", expand_measurements=True)
# → "a ye kilogaramu duuru san"

# WER/CER presets enable all expansions by default
config = BambaraNormalizerConfig.for_wer_evaluation()
# expand_numbers=True, expand_dates=True, expand_times=True, expand_measurements=True

Installation

pip install bambara-text-normalizer

or from github

pip install git+https://github.com/sudoping01/bambara-text-normalization.git

Changelog

Added

  • measurements.py module with weight, length, volume, area TN/ITN
  • expand_measurements configuration option
  • measurement_to_bambara() function
  • bambara_to_measurement() function
  • normalize_measurements_in_text() function
  • denormalize_measurements_in_text() function
  • is_measurement_word() utility function
  • get_unit_category() utility function
  • 30 measurement normalization tests

Changed

  • WER/CER presets now enable expand_measurements=True
  • Updated README with Measurement Normalization section

Full Documentation

See README.md for complete documentation.


Citation

@software{bambara_tn_itn,
  title = {Bambara Text Normalization and Inverse Text Normalization (TN/ITN) Toolkit},
  author = {MALIBA-AI},
  year = {2025},
  version = {2.4.0},
  url = {https://github.com/sudoping01/bambara-text-normalization},
  note = {A comprehensive TN/ITN framework for Bambara (Bamanankan) with support for numbers, dates, times, measurements, and ASR evaluation}
}

MALIBA-AI 🇲🇱

# Bambara Text Normalizer v1.0.1