Skip to content

Releases: dimitree54/nl_processing

v1.0.4

Choose a tag to compare

@dimitree54 dimitree54 released this 10 Mar 07:36

Bug Fix

Fix: WordExtractor prompt/schema contract inconsistency (#2)

The Dutch extraction prompt instructed the LLM to extract phraseological constructions as single units, but PartOfSpeech had no phrase value. When the LLM returned word_type="phrase", Pydantic validation failed and the entire extraction raised APIError — breaking save-to-dictionary flows in production.

Changes

  • Added PHRASE = "phrase" to PartOfSpeech enum in core.models
  • Added phrase to the allowed types list in the NL extraction prompt
  • Added a few-shot example demonstrating phraseological unit extraction
  • Regenerated nl.json prompt asset
  • Added prompt-schema consistency tests to prevent future drift

v1.0.2

Choose a tag to compare

@dimitree54 dimitree54 released this 08 Mar 21:38

Highlights

  • migrate repository and package docs to the canonical module-spec format
  • add shared storage ports in core (ScoredPairProvider, RemoteProgressSyncPort) and the WordPairSnapshot sync DTO
  • decouple database_cache from a hard-coded remote progress store while keeping ExerciseProgressStore as the default adapter
  • align sampling, database, and cache docs and contracts around the new protocol-based storage boundary

v1.0.1

Choose a tag to compare

@dimitree54 dimitree54 released this 08 Mar 17:32

Major release for the multi-package monorepo migration.

Highlights:

  • repository reorganized into package-local projects under packages/*
  • each package now has its own pyproject.toml, ruff.toml, pytest.ini, tests/, and Makefile
  • root make check now delegates into package-local make check
  • shared DTOs such as WordPair and ScoredWordPair now live only in nl_processing.core.models
  • legacy re-exports for shared models were removed
  • aggregate root build was updated to package the multi-root layout correctly

Breaking changes:

  • shared model imports must use nl_processing.core.models
  • old single-tree repo layout and root test layout are no longer the source of truth

v0.5.0

Choose a tag to compare

@dimitree54 dimitree54 released this 08 Mar 16:23

What's New in 0.5.0

New Features

  • WordSampler now supports swappable scored data sources via scored_store, enabling the same sampling logic to run against either remote ExerciseProgressStore or local DatabaseCacheService
  • Added ScoredPairProvider Protocol to formalize the minimal sampler dependency: get_word_pairs_with_scores()
  • Preserved backward compatibility: existing WordSampler(...) usage still constructs ExerciseProgressStore internally when scored_store is not provided

Documentation

  • Updated sampling PRD and architecture docs to define Protocol-based injection and cache-backed sampling
  • Added sprint plan under docs/sprints/sampling-swappable-store/

Testing and Quality

  • Added unit coverage for injected-store construction, Protocol conformance, and sampling via injected stores
  • Added ScoredPairProvider to the vulture whitelist
  • make check passes fully

v0.4.0

Choose a tag to compare

@dimitree54 dimitree54 released this 08 Mar 15:19

What's New in 0.4.0

New Module: database_cache

  • Local-first SQLite cache for the vocabulary practice loop — serves reads and accepts writes locally, syncs with remote Neon in the background
  • Full public API: DatabaseCacheService with init(), get_words(), get_word_pairs_with_scores(), record_exercise_result(), refresh(), flush(), get_status()
  • Stale-while-revalidate refresh policy — fresh data served immediately, background refresh when stale
  • Transactional outbox pattern — local score writes + event recording in a single SQLite transaction
  • Idempotent remote replay — event IDs prevent double-application of score deltas on flush
  • Atomic snapshot rebuild — refresh replaces the full snapshot and reapplies pending events in one transaction
  • 4 SQLite tables: cached_word_pairs, cached_scores, pending_score_events, cache_metadata
  • Comprehensive test suite: 44 tests (28 unit, 13 integration, 3 E2E against real Neon)

Auto-Flush

  • record_exercise_result() now automatically triggers a fire-and-forget background flush via asyncio.create_task
  • flush() remains available as a public method for explicit use
  • Background auto-flush uses skip-if-running semantics; explicit flush() waits for completion

Breaking Changes

  • Removed CachedDatabaseService from database module (legacy, superseded by database_cache)

Bug Fixes

  • Fixed flush race condition: explicit flush() calls now wait for any running background flush to complete instead of silently skipping. Previously, calling flush() right after record_exercise_result() could return as a no-op if the auto-flush held the lock.
  • Fixed apply_score_delta to validate delta values (+1/-1) and exercise types (FR32)
  • Made apply_score_delta atomic (single-transaction check+increment+mark)
  • Added missing FR21 implementation: get_words() logs warning for untranslated words

Documentation

  • All module docs synced with implementation — removed phantom Protocol references, fixed architecture diagrams
  • Shared docs/architecture.md and docs/prd.md updated to reference database_cache and remove CachedDatabaseService
  • All doc files trimmed to ≤200 lines (repo convention)
  • PRD FR38-41 updated to document atomic score replay and delta validation
  • Architecture doc corrected: sampling reads directly from database, not through database_cache
  • Removed stale database_cached/ directories from prior module rename

Other Changes

  • Added aiosqlite dependency
  • Updated vulture_whitelist.py for new public API symbols
  • Removed obsolete sampling integration tests and latency benchmark test

v0.3.0

Choose a tag to compare

@dimitree54 dimitree54 released this 07 Mar 19:43

What's New in 0.3.0

Breaking Changes

  • ExerciseProgressStore constructor now requires exercise_types: list[str] parameter
  • ExerciseProgressStore.increment() takes source_word_id: int instead of source_word: Word
  • ExerciseProgressStore.get_word_pairs_with_scores() no longer accepts exercise_types parameter (uses configured set)
  • ScoredWordPair model now includes source_word_id: int field

New Features

  • Per-exercise-type score tables: Exercise statistics are now stored in separate database tables per exercise type (e.g., user_word_exercise_scores_nl_ru_flashcard), improving isolation and making table ownership explicit
  • Cache-support APIs: New export_remote_snapshot() and apply_score_delta(event_id, ...) methods on ExerciseProgressStore for the planned database_cache module
  • Idempotent score replay: apply_score_delta uses event IDs to prevent double-application of remote score deltas
  • database_cache module documentation: Full PRD, architecture, and product brief for the upcoming local-first cache module

Bug Fixes

  • Fixed DatabaseService.add_words() to use the word's actual language attribute for table routing instead of always defaulting to the source language

Other Changes

  • CachedDatabaseService marked as legacy (superseded by planned database_cache)
  • NeonBackend exercise methods extracted to _neon_exercise.py to stay under 200-line limit
  • Updated README with database_cache recommendation and capabilities overview
  • sampling module updated to pass exercise_types to ExerciseProgressStore

v0.2.2

Choose a tag to compare

@dimitree54 dimitree54 released this 05 Mar 21:58
  • add interjection to the shared PartOfSpeech taxonomy and align translate_word / extract_words prompts
  • update tests for interjection handling and rotated vocabulary expectations
  • stop tracking Python cache files and ignore generated build artifacts in release checks

v0.2.1

Choose a tag to compare

@dimitree54 dimitree54 released this 05 Mar 21:20

Fix packaging of runtime prompt JSON artifacts.

  • include prompt JSON files in built wheels/sdists for all prompt-backed modules
  • add regression tests that verify wheel contents for extract_text_from_image, extract_words_from_text, translate_text, and translate_word
  • fix WordExtractor(Language.NL) crash from missing packaged prompt file

v0.2.0

Choose a tag to compare

@dimitree54 dimitree54 released this 05 Mar 17:53

What's New

Database module (database)

  • DatabaseService — persist words and translations to Neon PostgreSQL
  • CachedDatabaseService — LRU-cached variant for read-heavy workloads
  • ExerciseProgressStore — per-user per-exercise score tracking
  • Neon backend with asyncpg, abstract backend ABC for extensibility
  • Testing utilities (drop_all_tables, reset_database, helpers)

Sampling module (sampling)

  • WordSampler — weighted word sampling with adversarial distractor selection

Docs

  • Added DATABASE_URL to ENV_VARS.md
  • Added sampling module to README
  • Updated database module description in README
  • NEON.md with full Neon PostgreSQL setup instructions

Tests

  • 133 unit tests passing
  • Integration and e2e tests for database and sampling modules