Skip to content

v0.4.0

Choose a tag to compare

@github-actions github-actions released this 30 Jul 16:20
164fdf6

numpy is now the only runtime dependency, and a trained map can be saved without pickle.

Results change in two places. Linear initialization is corrected for data far from the origin,
where 0.3.0 was wrong by up to 5.8%; and anyone who imported pandas or scikit-learn transitively
through this package must now depend on them directly. Everything else is additive: no public name or
signature is removed, and mode="batch" and the other plain strings keep working.

To reproduce a figure made with an earlier version, pin that version.

Added

  • save_npz and load_npz, so a trained map can be kept without reaching for pickle. One
    .npz holds the models plus a JSON block with the configuration, the seed, the generator state
    and
    the last training report. No pickle on either side, so loading one cannot execute code.

    A reloaded map continues the same random stream, so training it further gives what never
    stopping would have given. Both the seed and the generator's current state are saved: the seed
    alone
    would restart the stream and a resumed run would quietly diverge from an uninterrupted one.

    Strategies are saved by name and resolved through registries. A map built from the shipped
    functions
    round-trips completely; one built with your own function records the name for provenance and
    raises
    ArtifactError on load, naming the argument to pass it back through. A functools.partial is
    treated as custom, so partial(exponential_decay, factor=3.0) cannot silently reload as
    factor=2.0.

    On safety: allow_pickle=False is passed explicitly, names resolve only through the registries,
    and
    the member list, JSON, format version and weight shape are all validated before a map is built.
    What
    that buys is "cannot execute code", not "safe to load anything". An .npz is a zip, so a hostile
    file can still attempt resource exhaustion. docs/save-and-load.md states both halves.

  • SOM.last_report, a frozen TrainingReport: mode, iterations, sample count, seed, the final
    learning rate and radius, quantization error, the python_som and NumPy versions, and wall time.
    train() still returns the quantization error, so nothing that used the return value changes.
    final_learning_rate is None for batch training, which has no step size: Eq. (8) is a weighted
    mean, and reporting the unused initial value would read as though it had been applied.

  • SOMConfig, the serialisable description of a map, available as som.config().

  • DECAY_FUNCTIONS and DISTANCE_FUNCTIONS registries with resolve_decay and
    resolve_distance, mirroring NEIGHBORHOOD_FUNCTIONS. These names are written into artifacts, so
    they are public API from 0.4.0.

  • An analysis extra with pinned bandit, pip-audit and pylint, for the deeper review passes.
    Not wired into any gate; ruff and mypy remain the enforced checks. CI names the extras it needs
    rather than using --all-extras, so these are not installed on every matrix entry.

  • Enums for every string-valued option: TrainingMode, Neighborhood, WeightInit and
    SampleMode. Each member is a str, so TrainingMode.BATCH and "batch" are
    interchangeable: equal, same hash, same JSON, usable as the same dictionary key. Nothing that
    accepted a string stops accepting one.

    The parameters are typed as the enum or the exact set of valid strings, so a type checker now
    rejects mode="bacth" while accepting both mode="batch" and mode=TrainingMode.BATCH. If you
    build an option from configuration, annotate it with the matching TrainingModeStr alias or call
    TrainingMode(value) to validate it at runtime.

    Plain strings are deprecated but do not warn yet. They warn in 0.5.0 and are removed in 1.0.0.
    Warning now would fire on mode="batch", which is what the documentation showed until this
    release, so the written notice comes first. See docs/options-and-types.md.

  • Protocols for the three replaceable strategies: NeighborhoodFunction, DecayFunction and
    DistanceFunction, plus KernelFunction for the batch kernel. Structural, so nothing inherits
    from them and every existing callable already satisfies its protocol; their parameters are
    positional-only, so your own parameter names are your own. A type checker now checks a
    user-supplied strategy against a named contract instead of a bare Callable.

Changed

  • learning_rate is validated, having been unchecked. A non-positive or non-finite rate raises
    ValueError: 0 freezes every model so training completes and changes nothing, and -1 moves
    models away from the samples they match, taking the quantization error from 0.0 to 11.7. Both were
    previously accepted in silence.

    A rate above 1 emits UserWarning rather than raising. It overshoots and oscillates but does not
    necessarily diverge: at alpha = 5 with decay disabled the largest weight stayed at 3.61, because
    the neighborhood damps the correction away from the winner. Kohonen gives no upper bound, so
    rejecting it would invent a limit the sources do not.

  • The package is now a pure functional core with a thin shell around it. python_som._core holds
    every numeric decision as functions over NumPy arrays and imports nothing but NumPy;
    python_som._convert adapts pandas at the boundary; python_som._som keeps the state, the
    validation and the training loops. No public name, signature or numerical result changes:
    trained weights are bit-identical to 0.3.0 for every combination of training mode and neighborhood
    function, which is asserted rather than assumed.

    The boundary is machine-checked, not merely documented: ruff's TID251 bans pandas and
    scikit-learn outside the two shell modules that exist to adapt them, and reports the reason at the
    point of violation. architecture-profile.toml records the style.

  • The two update rules are now pure functions returning new arrays. An earlier note claimed this
    was also faster. It is not, and the claim is withdrawn. Measured with interleaved arms and
    medians, the pure form is about 10% slower on small maps and indistinguishable above roughly
    100x100. The cost buys functions testable without constructing a network, and is single-digit
    milliseconds over a 10,000-iteration run on a 20x20 map. benchmarks/bench_update.py is the
    corrected measurement.

Performance

  • Batch training is 1.2x to 1.5x faster, with identical results. Eq. (8) needs the neighborhood
    between every pair of nodes, and the previous implementation got it by evaluating the neighborhood
    function once per node, once per iteration, which is 1600 full-grid evaluations per iteration on a 40x40
    map. Profiling made that 42% of batch training, more than the contraction it feeds.

    It is unnecessary, because a neighborhood depends only on the offset between two nodes and never
    on where the winner sits. One kernel over every reachable offset serves the whole grid, and
    each node's neighborhood is a slice of it, as a view rather than a copy. The kernel costs 4xy floats,
    111 KB on a 60x60 map, against the 800 MB a full (x, y, x, y) tensor would need.

    The offset-only dependence holds on a torus as well as a flat grid, because making the fold depend
    on the offset alone is exactly what the minimum-image convention does. It holds per axis, so mixed
    maps (one axis wrapping, one not) need no special case.

    Results do not change: trained weights are bit-identical for all 72 working combinations of
    training mode, neighborhood function, initializer and cyclic setting, and the kernel is checked
    against per-node evaluation across 40,832 combinations at exactly 0.0. The gaussian gains more
    than the bubble, whose per-node form is cheaper to begin with. benchmarks/bench_batch.py is the
    measurement.

    Stepwise training is unaffected: it already evaluates the neighborhood once per iteration, so
    there is nothing to amortize.

Fixed

  • Linear initialization was inaccurate for data far from the origin. Through 0.3.0 its PCA went
    to scikit-learn's auto solver, which since 1.5 selects covariance_eigh when samples outnumber
    features. It forms the covariance matrix, and squaring the data squares its condition number. On
    (150, 4) data offset by 1e7, the second explained variance was wrong by 5.8%, and the
    resulting models differed from the correct ones by 2.43 against a total model spread of 2.0: an
    error larger than the structure being initialized.

    The NumPy implementation decomposes the centred matrix directly and agrees with a longdouble
    reference to ~1e-15. Data offset far from the origin is not exotic: timestamps, easting/northing
    coordinates and absolute sensor readings all look like it.

    This changes results. For data near the origin the difference is floating-point noise (~1e-15,
    and auto_dimensions is unaffected because it standardizes first). Far from the origin it is
    large, and 0.4.0 is the correct one.

  • _train_stepwise looked up array[index] twice per iteration, allocating the sample twice on the
    hot loop. Found by profiling the refactor rather than by reading it.

Removed

  • pandas and scikit-learn are no longer required. numpy is the only runtime dependency.
    Measured on Linux/CPython 3.12, a fresh install drops from 333 MB across 10 packages to 69 MB
    across 1, a 264 MB reduction and 79% of the payload, since the two also pulled in scipy, joblib,
    narwhals, python-dateutil, six and threadpoolctl.

    Nothing is lost, and input support is wider. pandas was used for one isinstance check before
    calling .to_numpy(); np.asarray already does that via the __array__ protocol. Because
    that is a protocol rather than a library, polars, pyarrow, xarray and CuPy objects now work too,
    without this package knowing they exist.

    scikit-learn supplied one PCA and one z-score, now about twenty lines of np.linalg.svd. Both
    libraries remain in the dev extra, where tests/test_linalg_matches_sklearn.py re-derives every
    fit both ways on every CI run rather than trusting a one-off measurement. pip install python-som[examples] restores the previous install set for anyone relying on it.

    If you imported pandas or scikit-learn transitively through this package, add them to your own
    dependencies. That reliance was always an accident of packaging.

  • The batch denominator's 1e-12 tolerance, replaced by > 0. The constant had no source, and it
    guarded a condition that cannot occur: every term of sum_j n_j h_ji is non-negative, because
    batch training rejects signed neighborhoods and only registered names resolve, so a sum of
    non-negative floats is zero exactly when every term is zero. The 0.3.0 entry below describes it as
    "compared against a tolerance rather than exact zero"; that was the right instinct applied to a
    problem that was not there.