Skip to content

Choose a tag to compare

@jzstark jzstark released this 17 Nov 13:56
· 2 commits to main since this release

CHANGES:

  • Build: do not inherit OpenMP flags from the pkg-config files of OpenBLAS,
    CBLAS or LAPACKE unless OWL_ENABLE_OPENMP=1. An OpenBLAS built with
    OpenMP (Homebrew, FreeBSD) made Owl compile its #pragma omp code without
    linking an OpenMP runtime (undefined __kmpc_* symbols, #695) or fail with
    compilers that reject -fopenmp (#684).
  • Build: work around Apple Clang 21 (Xcode 26) crashing on arm64 while
    vectorising the modf loops of owl_ndarray_maths_stub.c: with Clang,
    modf is called through a function that is not inlined.
  • Build: keep the optional optimisation flags of the default configuration
    only if the C compiler accepts them without a warning, so that GCC-only
    flags are dropped with Clang and -march=native with compilers that
    reject it. Flags with GCC are unchanged.
  • Fix resuming a compiled network (Owl_neural_compiler.train ~state): the
    weights were always initialised again and the optimiser state (gradient
    history, momentum, Adam's moments) started from zero. train and
    compile_deep take ?init_model (default true, as in Graph.train);
    pass ~init_model:false with ~state to resume. The optimiser state is
    now saved in the returned state and restored from it. Training an already
    compiled network a second time no longer fails in Graph.init.
  • Fix the learning rate of compiled networks (Owl_neural_compiler.train):
    the static graph used the rate of step 999 for every step, so Adam's bias
    correction was wrong and Decay, Exp_decay and Schedule were constant;
    Adam also still divided by the gradient. The step-dependent factor is now a
    scalar graph input set before every step (assign_learning_rate), using the
    new Learning_Rate.scale and Learning_Rate.unscaled_update. Compiled
    training now matches eager training step for step.
  • Fix NaN in cross-entropy when a prediction underflows to 0, e.g. from a
    saturated softmax: Algodiff.Maths.cross_entropy (and so
    Loss.Cross_entropy) and Ndarray.cross_entropy' raise predictions below
    the smallest positive normal float to that value. Results for all other
    predictions are unchanged bit for bit; NaN predictions still propagate.
  • Change Regularisation.L2norm a to a * ||w||^2 (it was a * ||w||), and
    the L2 term of Regularisation.Elastic_net likewise. The plain norm has a
    NaN gradient at w = 0, so training a network with L2 regularisation
    turned every parameter into NaN, starting from the zero-initialised
    biases. Regression.svm uses the squared penalty as a result.
  • Fix ~keep_dims:true without ~axis in sum, prod, min, max,
    minmax, mean, median, var, std, sem, l1norm, l2norm,
    l2norm_sqr and vecnorm (and sum, min, max in owl-base and
    Algodiff.Maths.sum): it now keeps every dimension with size 1 instead of
    returning shape [|1|] (#621). Calls that do not pass keep_dims still
    return [|1|], so default results are unchanged.
  • Fix Linalg.det and Linalg.logdet on singular matrices: they return 0
    and neg_infinity instead of raising Failure "LAPACKE: i" (#249). The
    pure-OCaml Owl_base_linalg_generic.det returned wrong values (e.g. -4 for
    [[1, 2], [2, 4]]) because the LU pivot search reused a stale row when a
    column was zero; it now returns 0, also for matrices with a zero row.
  • Fix Stopping.Const: Checkpoint.run overwrote the stop flag set by the
    stopping criterion, so training never ended early. It now stops at the
    first batch whose loss is below the threshold, in minimise_weight,
    minimise_fun, minimise_network and minimise_compiled_network.
  • Fix Adam: the update no longer divides by the gradient, so elements with a
    zero gradient keep moving with their first moment and tiny gradients no
    longer give infinite steps. Applies to minimise_weight, minimise_fun,
    minimise_network and Stepper; other learning-rate strategies are
    unchanged bit for bit. Adam results with non-zero gradients may differ in
    the last ulp. New Learning_Rate.update returns the weight update.
  • Fix Regression: ols and ridge are solved directly via the SVD,
    lasso and elastic_net by coordinate descent, so results are exact
    rather than the outcome of 1000 Adagrad steps (#340). Penalties no longer
    apply to the intercept, and the L2 penalty of ridge and elastic_net is
    now the squared norm. logistic now models sigmoid (x w + b) with targets
    in [0, 1] and is solved by Newton's method; it used to take the log of the
    raw linear output. exponential uses variable projection and fits growing
    exponentials (#650). svm no longer prints its training log. Documentation
    now states the actual defaults (i = false, alpha = 0.001).
  • Add standalone single-step optimiser Optimise.Stepper
  • Add unit tests for Optimise and Regression
  • Fix exponpow_rvs: it called the standard Gaussian sampler with the scale
    as its only argument; it now samples a Gaussian with that scale (#694,
    @maxkuttner)
  • Fix the documentation of the gumbel2 distributions: the second parameter
    is a shape, not a location (#696, @tristan-f-r)
  • Publish the documentation from the gh-pages branch; the generated HTML is
    no longer committed in docs/
  • Run the tests in CI on Ubuntu and on macOS (arm64 and Intel)