Skip to content

v0.0.3

Choose a tag to compare

View all tags
@github-actions github-actions released this 23 Dec 06:32
· 1270 commits to main since this release
v0.0.3
a0621e7

Added

  • Inference Serving:

    • Production-ready REST API with FastAPI
    • Streaming support via Server-Sent Events (SSE)
    • Advanced sampling strategies (nucleus sampling/top-p, repetition penalty)
    • Prometheus metrics endpoint for monitoring
    • API key authentication (X-API-Key header)
    • Structured logging with python-json-logger
    • Real PyTorch model weights loading from checkpoint files
    • Pickled tokenizer object loading support

  • Component Registry:

    • Automatic component registration system (ComponentRegistry)
    • Core components (MHA, MLP, MoE) auto-registered via side-effect imports
    • Prevents "component not found" errors in simplified scripts

  • Data Abstraction:

    • Formalized BaseTokenizer protocol
    • BaseDataModule abstraction for flexible data handling
    • Environment variable configuration support (e.g., LLM_TRAINING__EPOCHS)

  • Testing & CLI:

    • --num-samples flag in train.py for rapid regression testing
    • Scheduler edge case tests (test_scheduler_edge_cases.py)
    • Validation logging tests (test_engine_logging.py)
    • Component registry tests (test_init.py)
    • Model loading verification tests
    • Auto-device detection in training scripts (prioritizes CUDA)

  • Documentation:

    • Comprehensive usage guide (docs/usage.md)
    • Architecture documentation (docs/architecture.md)
    • Engineering documentation (ADRs, PR templates, FAQ)
    • VS Code configuration and extensions

Changed

  • Architecture Modernization:

    • Migrated to Pydantic v2 (BaseSettings, BaseModel) for configuration
    • Fully typed and validated configuration system
    • CLI migration from argparse to typer for better UX

  • Naming Standardization:

    • Unified ffn_hidden_size → intermediate_size across codebase
    • Standardized input parameter x → hidden_states in forward methods
    • Applied to MLP, LayerNorm, RMSNorm, DecoderModel, TransformerBlock
    • Updated all 309 tests to reflect API changes

  • Code Quality:

    • Standardized punctuation in documentation (full-width → half-width)
    • Improved type hints and documentation comments
    • Refactored TransformerBlock.forward for clarity

Fixed

  • Core Bugs:

    • CosineAnnealingLR T_max calculation when epochs == warmup_epochs (ZeroDivisionError)
    • TrainingEngine validation logging crash when gradient_norms is empty (IndexError)
    • PAD token generation issue in inference (logits masking)
    • SyntheticDataModule prefetch_factor handling with num_workers=0
    • TransformerBlock shared norm instance bug (independent norm1/norm2)
    • Scheduler/optimizer step order warnings in tests
    • PositionalEncoding support for start_pos in incremental generation
    • MLP SwiGLU operation order for numerical consistency
    • Prompt truncation respecting max_seq_len with new tokens
    • Auto AMP dtype resolution for CPU-only environments

  • Registry & Imports:

    • Package auto-registration via import llm
    • Component not found errors in simplified execution
Assets 2
Loading