Second alpha. Three breaking changes — read the first section before upgrading.
Changed
- Breaking.
box_plotandplot_prtake a pandas, Polars, or Spark DataFrame, like the rest of the package. Theplotextra declares pandas and is sufficient on top of any backend: a Polars frame is collected without PyArrow when PyArrow is absent, sincereplicas[polars,plot]pulls none andpolars.DataFrame.to_pandasgoes through Arrow.box_plotwas pandas-only andplot_prwas Spark-only, so the two plot helpers demanded opposite backends and a pandas user could not callplot_prat all. Each one now reduces the data on its own backend and collects only the result. - Breaking. The
sparkextra requires PySpark 3.5, up from 3.3.pr_bandusesF.percentile, added in 3.5.percentile_approxreturns a different order statistic — on 100 replicas it put the 5th-percentile band edge 2.4% off the pandas and Polars value, which is a visible shift in a plotted band, not a rounding difference. Spark 3.3 and 3.4 are both past end of life. - Breaking.
confusion_table,calculate_pr, andattakebyinstead ofgroup_by. Every public function that partitions a computation by columns now uses the same name, matchingsampleandbootstrapand pandas' owngroupby(by=...). Positional calls are unaffected; agroup_by=keyword call raisesTypeError. The type aliasreplicas.metrics.GroupByis nowByColumns. - The
replicacolumn is a 32-bit integer on all three backends. pandas previously producedint64and PolarsInt64. - Every public function documents its arguments, its return value, and the errors a caller can act on.
replicas.plottinguses the same docstring convention as the rest of the package and namesseaborn.FacetGridas what both helpers return.sampledocuments theround(group_size * fraction)draw size, including fractions above 1, and both samplers document that the uniquenessorder_byneeds for a reproducible seeded Spark draw is the caller's responsibility and is not verified. atdocuments that its "lowest qualifying threshold" rule suits a metric that does not increase as the threshold falls, such asprecision. Arecalltarget degenerates to the group's minimum threshold.confusion_tabledocuments that the non-null and mutually-exclusive conditions on its indicator columns are the caller's responsibility.- The metric functions and
bootstrapdocument the Spark cost of an emptyby. - The README defines "replica" once at first use, and states the limitation of a percentile-bootstrap band as asymptotic coverage with a finite-sample shortfall, instead of claiming the true interval is always the wider one.
bootstrappoints at the README for the NaN exception to the cross-backend parity guarantee.
Added
pr_band(df, by=..., ci=..., recall_round=...), the reduction behindplot_pr, as a public metric function on all three backends. It returns the original curve and a pointwise quantile band across the replicas. A null grouping value is one group, as it is everywhere else in the module: the three backends reach that through one grouped aggregation rather than a join, because they disagree on whether a join matches null keys.- A
notebookextra and a CI job that executesexamples/quickstart.ipynband compares its outputs with the committed ones.
Fixed
bootstraprestores the Spark checkpoint directory the caller had configured instead of leaving its own in place for the rest of the session. Spark offers no per-call checkpoint path, so the read/set/checkpoint/restore sequence runs under a process-wide lock: two concurrentbootstrapcalls can otherwise checkpoint into each other's directory. A restore that fails raises aRuntimeWarningrather than passing silently. Known limitation: Spark reports a checkpoint directory as<root>/<generated-id>and setting that value back appends another id, so a directory configured outsidereplicasends up one generated level deeper the first time it is restored. Later calls reuse the remembered root and the depth holds.- The local checkpoint fallback is scoped to the current user rather than a fixed
/tmp/replicas/that the first user on a machine takes ownership of. box_plotexplains that it needs areplicacolumn instead of raising a bare pandasKeyError.plot_prrejects acioutside(0, 1]up front, instead of failing later insidepercentile_approx.
Full changelog: v0.1.0...v0.2.0