Repository navigation
Releases: dimitree54/nl_processing
Releases · dimitree54/nl_processing
Release list
v1.0.4
Bug Fix
Fix: WordExtractor prompt/schema contract inconsistency (#2)
The Dutch extraction prompt instructed the LLM to extract phraseological constructions as single units, but PartOfSpeech had no phrase value. When the LLM returned word_type="phrase", Pydantic validation failed and the entire extraction raised APIError — breaking save-to-dictionary flows in production.
Changes
- Added
PHRASE = "phrase"toPartOfSpeechenum incore.models - Added
phraseto the allowed types list in the NL extraction prompt - Added a few-shot example demonstrating phraseological unit extraction
- Regenerated
nl.jsonprompt asset - Added prompt-schema consistency tests to prevent future drift
v1.0.2
Highlights
- migrate repository and package docs to the canonical module-spec format
- add shared storage ports in core (
ScoredPairProvider,RemoteProgressSyncPort) and theWordPairSnapshotsync DTO - decouple
database_cachefrom a hard-coded remote progress store while keepingExerciseProgressStoreas the default adapter - align
sampling,database, and cache docs and contracts around the new protocol-based storage boundary
v1.0.1
Major release for the multi-package monorepo migration.
Highlights:
- repository reorganized into package-local projects under
packages/* - each package now has its own
pyproject.toml,ruff.toml,pytest.ini,tests/, andMakefile - root
make checknow delegates into package-localmake check - shared DTOs such as
WordPairandScoredWordPairnow live only innl_processing.core.models - legacy re-exports for shared models were removed
- aggregate root build was updated to package the multi-root layout correctly
Breaking changes:
- shared model imports must use
nl_processing.core.models - old single-tree repo layout and root test layout are no longer the source of truth
v0.5.0
What's New in 0.5.0
New Features
WordSamplernow supports swappable scored data sources viascored_store, enabling the same sampling logic to run against either remoteExerciseProgressStoreor localDatabaseCacheService- Added
ScoredPairProviderProtocol to formalize the minimal sampler dependency:get_word_pairs_with_scores() - Preserved backward compatibility: existing
WordSampler(...)usage still constructsExerciseProgressStoreinternally whenscored_storeis not provided
Documentation
- Updated
samplingPRD and architecture docs to define Protocol-based injection and cache-backed sampling - Added sprint plan under
docs/sprints/sampling-swappable-store/
Testing and Quality
- Added unit coverage for injected-store construction, Protocol conformance, and sampling via injected stores
- Added
ScoredPairProviderto the vulture whitelist make checkpasses fully
v0.4.0
What's New in 0.4.0
New Module: database_cache
- Local-first SQLite cache for the vocabulary practice loop — serves reads and accepts writes locally, syncs with remote Neon in the background
- Full public API:
DatabaseCacheServicewithinit(),get_words(),get_word_pairs_with_scores(),record_exercise_result(),refresh(),flush(),get_status() - Stale-while-revalidate refresh policy — fresh data served immediately, background refresh when stale
- Transactional outbox pattern — local score writes + event recording in a single SQLite transaction
- Idempotent remote replay — event IDs prevent double-application of score deltas on flush
- Atomic snapshot rebuild — refresh replaces the full snapshot and reapplies pending events in one transaction
- 4 SQLite tables:
cached_word_pairs,cached_scores,pending_score_events,cache_metadata - Comprehensive test suite: 44 tests (28 unit, 13 integration, 3 E2E against real Neon)
Auto-Flush
record_exercise_result()now automatically triggers a fire-and-forget background flush viaasyncio.create_taskflush()remains available as a public method for explicit use- Background auto-flush uses skip-if-running semantics; explicit
flush()waits for completion
Breaking Changes
- Removed
CachedDatabaseServicefromdatabasemodule (legacy, superseded bydatabase_cache)
Bug Fixes
- Fixed flush race condition: explicit
flush()calls now wait for any running background flush to complete instead of silently skipping. Previously, callingflush()right afterrecord_exercise_result()could return as a no-op if the auto-flush held the lock. - Fixed
apply_score_deltato validate delta values (+1/-1) and exercise types (FR32) - Made
apply_score_deltaatomic (single-transaction check+increment+mark) - Added missing FR21 implementation:
get_words()logs warning for untranslated words
Documentation
- All module docs synced with implementation — removed phantom Protocol references, fixed architecture diagrams
- Shared
docs/architecture.mdanddocs/prd.mdupdated to referencedatabase_cacheand removeCachedDatabaseService - All doc files trimmed to ≤200 lines (repo convention)
- PRD FR38-41 updated to document atomic score replay and delta validation
- Architecture doc corrected: sampling reads directly from
database, not throughdatabase_cache - Removed stale
database_cached/directories from prior module rename
Other Changes
- Added
aiosqlitedependency - Updated
vulture_whitelist.pyfor new public API symbols - Removed obsolete sampling integration tests and latency benchmark test
v0.3.0
What's New in 0.3.0
Breaking Changes
ExerciseProgressStoreconstructor now requiresexercise_types: list[str]parameterExerciseProgressStore.increment()takessource_word_id: intinstead ofsource_word: WordExerciseProgressStore.get_word_pairs_with_scores()no longer acceptsexercise_typesparameter (uses configured set)ScoredWordPairmodel now includessource_word_id: intfield
New Features
- Per-exercise-type score tables: Exercise statistics are now stored in separate database tables per exercise type (e.g.,
user_word_exercise_scores_nl_ru_flashcard), improving isolation and making table ownership explicit - Cache-support APIs: New
export_remote_snapshot()andapply_score_delta(event_id, ...)methods onExerciseProgressStorefor the planneddatabase_cachemodule - Idempotent score replay:
apply_score_deltauses event IDs to prevent double-application of remote score deltas database_cachemodule documentation: Full PRD, architecture, and product brief for the upcoming local-first cache module
Bug Fixes
- Fixed
DatabaseService.add_words()to use the word's actuallanguageattribute for table routing instead of always defaulting to the source language
Other Changes
CachedDatabaseServicemarked as legacy (superseded by planneddatabase_cache)NeonBackendexercise methods extracted to_neon_exercise.pyto stay under 200-line limit- Updated README with
database_cacherecommendation and capabilities overview samplingmodule updated to passexercise_typestoExerciseProgressStore
v0.2.2
- add interjection to the shared PartOfSpeech taxonomy and align translate_word / extract_words prompts
- update tests for interjection handling and rotated vocabulary expectations
- stop tracking Python cache files and ignore generated build artifacts in release checks
v0.2.1
Fix packaging of runtime prompt JSON artifacts.
- include prompt JSON files in built wheels/sdists for all prompt-backed modules
- add regression tests that verify wheel contents for extract_text_from_image, extract_words_from_text, translate_text, and translate_word
- fix WordExtractor(Language.NL) crash from missing packaged prompt file
v0.2.0
What's New
Database module (database)
DatabaseService— persist words and translations to Neon PostgreSQLCachedDatabaseService— LRU-cached variant for read-heavy workloadsExerciseProgressStore— per-user per-exercise score tracking- Neon backend with
asyncpg, abstract backend ABC for extensibility - Testing utilities (
drop_all_tables,reset_database, helpers)
Sampling module (sampling)
WordSampler— weighted word sampling with adversarial distractor selection
Docs
- Added
DATABASE_URLto ENV_VARS.md - Added
samplingmodule to README - Updated
databasemodule description in README - NEON.md with full Neon PostgreSQL setup instructions
Tests
- 133 unit tests passing
- Integration and e2e tests for database and sampling modules