v0.4.0
numpy is now the only runtime dependency, and a trained map can be saved without pickle.
Results change in two places. Linear initialization is corrected for data far from the origin,
where 0.3.0 was wrong by up to 5.8%; and anyone who imported pandas or scikit-learn transitively
through this package must now depend on them directly. Everything else is additive: no public name or
signature is removed, and mode="batch" and the other plain strings keep working.
To reproduce a figure made with an earlier version, pin that version.
Added
-
save_npzandload_npz, so a trained map can be kept without reaching forpickle. One
.npzholds the models plus a JSON block with the configuration, the seed, the generator state
and
the last training report. No pickle on either side, so loading one cannot execute code.A reloaded map continues the same random stream, so training it further gives what never
stopping would have given. Both the seed and the generator's current state are saved: the seed
alone
would restart the stream and a resumed run would quietly diverge from an uninterrupted one.Strategies are saved by name and resolved through registries. A map built from the shipped
functions
round-trips completely; one built with your own function records the name for provenance and
raises
ArtifactErroron load, naming the argument to pass it back through. Afunctools.partialis
treated as custom, sopartial(exponential_decay, factor=3.0)cannot silently reload as
factor=2.0.On safety:
allow_pickle=Falseis passed explicitly, names resolve only through the registries,
and
the member list, JSON, format version and weight shape are all validated before a map is built.
What
that buys is "cannot execute code", not "safe to load anything". An.npzis a zip, so a hostile
file can still attempt resource exhaustion.docs/save-and-load.mdstates both halves. -
SOM.last_report, a frozenTrainingReport: mode, iterations, sample count, seed, the final
learning rate and radius, quantization error, thepython_somand NumPy versions, and wall time.
train()still returns the quantization error, so nothing that used the return value changes.
final_learning_rateisNonefor batch training, which has no step size: Eq. (8) is a weighted
mean, and reporting the unused initial value would read as though it had been applied. -
SOMConfig, the serialisable description of a map, available assom.config(). -
DECAY_FUNCTIONSandDISTANCE_FUNCTIONSregistries withresolve_decayand
resolve_distance, mirroringNEIGHBORHOOD_FUNCTIONS. These names are written into artifacts, so
they are public API from 0.4.0. -
An
analysisextra with pinnedbandit,pip-auditandpylint, for the deeper review passes.
Not wired into any gate; ruff and mypy remain the enforced checks. CI names the extras it needs
rather than using--all-extras, so these are not installed on every matrix entry. -
Enums for every string-valued option:
TrainingMode,Neighborhood,WeightInitand
SampleMode. Each member is astr, soTrainingMode.BATCHand"batch"are
interchangeable: equal, same hash, same JSON, usable as the same dictionary key. Nothing that
accepted a string stops accepting one.The parameters are typed as the enum or the exact set of valid strings, so a type checker now
rejectsmode="bacth"while accepting bothmode="batch"andmode=TrainingMode.BATCH. If you
build an option from configuration, annotate it with the matchingTrainingModeStralias or call
TrainingMode(value)to validate it at runtime.Plain strings are deprecated but do not warn yet. They warn in 0.5.0 and are removed in 1.0.0.
Warning now would fire onmode="batch", which is what the documentation showed until this
release, so the written notice comes first. Seedocs/options-and-types.md. -
Protocols for the three replaceable strategies:NeighborhoodFunction,DecayFunctionand
DistanceFunction, plusKernelFunctionfor the batch kernel. Structural, so nothing inherits
from them and every existing callable already satisfies its protocol; their parameters are
positional-only, so your own parameter names are your own. A type checker now checks a
user-supplied strategy against a named contract instead of a bareCallable.
Changed
-
learning_rateis validated, having been unchecked. A non-positive or non-finite rate raises
ValueError:0freezes every model so training completes and changes nothing, and-1moves
models away from the samples they match, taking the quantization error from 0.0 to 11.7. Both were
previously accepted in silence.A rate above 1 emits
UserWarningrather than raising. It overshoots and oscillates but does not
necessarily diverge: atalpha = 5with decay disabled the largest weight stayed at 3.61, because
the neighborhood damps the correction away from the winner. Kohonen gives no upper bound, so
rejecting it would invent a limit the sources do not. -
The package is now a pure functional core with a thin shell around it.
python_som._coreholds
every numeric decision as functions over NumPy arrays and imports nothing but NumPy;
python_som._convertadapts pandas at the boundary;python_som._somkeeps the state, the
validation and the training loops. No public name, signature or numerical result changes:
trained weights are bit-identical to 0.3.0 for every combination of training mode and neighborhood
function, which is asserted rather than assumed.The boundary is machine-checked, not merely documented: ruff's
TID251bans pandas and
scikit-learn outside the two shell modules that exist to adapt them, and reports the reason at the
point of violation.architecture-profile.tomlrecords the style. -
The two update rules are now pure functions returning new arrays. An earlier note claimed this
was also faster. It is not, and the claim is withdrawn. Measured with interleaved arms and
medians, the pure form is about 10% slower on small maps and indistinguishable above roughly
100x100. The cost buys functions testable without constructing a network, and is single-digit
milliseconds over a 10,000-iteration run on a 20x20 map.benchmarks/bench_update.pyis the
corrected measurement.
Performance
-
Batch training is 1.2x to 1.5x faster, with identical results. Eq. (8) needs the neighborhood
between every pair of nodes, and the previous implementation got it by evaluating the neighborhood
function once per node, once per iteration, which is 1600 full-grid evaluations per iteration on a 40x40
map. Profiling made that 42% of batch training, more than the contraction it feeds.It is unnecessary, because a neighborhood depends only on the offset between two nodes and never
on where the winner sits. One kernel over every reachable offset serves the whole grid, and
each node's neighborhood is a slice of it, as a view rather than a copy. The kernel costs4xyfloats,
111 KB on a 60x60 map, against the 800 MB a full(x, y, x, y)tensor would need.The offset-only dependence holds on a torus as well as a flat grid, because making the fold depend
on the offset alone is exactly what the minimum-image convention does. It holds per axis, so mixed
maps (one axis wrapping, one not) need no special case.Results do not change: trained weights are bit-identical for all 72 working combinations of
training mode, neighborhood function, initializer and cyclic setting, and the kernel is checked
against per-node evaluation across 40,832 combinations at exactly0.0. The gaussian gains more
than the bubble, whose per-node form is cheaper to begin with.benchmarks/bench_batch.pyis the
measurement.Stepwise training is unaffected: it already evaluates the neighborhood once per iteration, so
there is nothing to amortize.
Fixed
-
Linear initialization was inaccurate for data far from the origin. Through 0.3.0 its PCA went
to scikit-learn'sautosolver, which since 1.5 selectscovariance_eighwhen samples outnumber
features. It forms the covariance matrix, and squaring the data squares its condition number. On
(150, 4)data offset by 1e7, the second explained variance was wrong by 5.8%, and the
resulting models differed from the correct ones by 2.43 against a total model spread of 2.0: an
error larger than the structure being initialized.The NumPy implementation decomposes the centred matrix directly and agrees with a
longdouble
reference to ~1e-15. Data offset far from the origin is not exotic: timestamps, easting/northing
coordinates and absolute sensor readings all look like it.This changes results. For data near the origin the difference is floating-point noise (~1e-15,
andauto_dimensionsis unaffected because it standardizes first). Far from the origin it is
large, and 0.4.0 is the correct one. -
_train_stepwiselooked uparray[index]twice per iteration, allocating the sample twice on the
hot loop. Found by profiling the refactor rather than by reading it.
Removed
-
pandas and scikit-learn are no longer required.
numpyis the only runtime dependency.
Measured on Linux/CPython 3.12, a fresh install drops from 333 MB across 10 packages to 69 MB
across 1, a 264 MB reduction and 79% of the payload, since the two also pulled in scipy, joblib,
narwhals, python-dateutil, six and threadpoolctl.Nothing is lost, and input support is wider. pandas was used for one
isinstancecheck before
calling.to_numpy();np.asarrayalready does that via the__array__protocol. Because
that is a protocol rather than a library, polars, pyarrow, xarray and CuPy objects now work too,
without this package knowing they exist.scikit-learn supplied one PCA and one z-score, now about twenty lines of
np.linalg.svd. Both
libraries remain in thedevextra, wheretests/test_linalg_matches_sklearn.pyre-derives every
fit both ways on every CI run rather than trusting a one-off measurement.pip install python-som[examples]restores the previous install set for anyone relying on it.If you imported pandas or scikit-learn transitively through this package, add them to your own
dependencies. That reliance was always an accident of packaging. -
The batch denominator's
1e-12tolerance, replaced by> 0. The constant had no source, and it
guarded a condition that cannot occur: every term ofsum_j n_j h_jiis non-negative, because
batch training rejects signed neighborhoods and only registered names resolve, so a sum of
non-negative floats is zero exactly when every term is zero. The 0.3.0 entry below describes it as
"compared against a tolerance rather than exact zero"; that was the right instinct applied to a
problem that was not there.