Skip to content

v2.0.0

Latest
Latest

Choose a tag to compare

View all tags
@MilesCranmer MilesCranmer released this 25 Aug 13:50
· 1 commit to master since this release
v2.0.0
df57080
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

2.0.0 (2026-08-25)

SymbolicRegression.jl 2.0.0 is the conclusion of a two-year transformation of the library from a fixed search pipeline into a modular PyTorch-like framework for symbolic learning, while still keeping its familiar v1 functionality and API. Operators can take any number of arguments, the search loop accepts plugins, and mutations and crossovers are first-class objects you can configure or replace. This also introduces the ability to pass "guesses" for the expressions, which mix into the populations throughout a run - which helps connect SR.jl to agentic coding loops. Finally, we get some major speed boosts from a huge reduction in allocations thanks to a reusable evaluation buffer.

Version: release 2.0.0, tag v2.0.0.

Highlights

Operators of any arity

Pass an OperatorEnum keyed by arity, and ternary or higher-degree operators become real tree nodes (#471, #472, #464). Before v2 the enum had exactly a unary and a binary slot, so piecewise logic such as a > 0 ? b : c had to be approximated by nests of binary operators that the search rarely found.

using SymbolicRegression
using SymbolicRegression: machine, fit!, predict, report

scalar_ifelse(a, b, c) = a > 0 ? b : c

X = randn(3, 100)
y = [X[1, i] > 0 ? 2*X[2, i] : X[3, i] for i in 1:100]

model = SRRegressor(
    operators=OperatorEnum(
        1 => (),
        2 => (+, -, *, /),
        3 => (scalar_ifelse,),
    ),
    niterations=35,
)
mach = machine(model, X', y)
fit!(mach)
report(mach)

binary_operators and unary_operators still work, and they remain mutually exclusive with operators=. Constraints are now degree-indexed: an N-argument operator takes an N-tuple, with -1 meaning unconstrained, and unary operators default to -1 as before. Operator and connection mutations, append/delete/rotate operations, crossover, simplification, constraint checking, and dimensional analysis operate over the generalized tree, whose node type became Node{T,D} in DynamicExpressions (#127). The lower-level swap_operands helper accepts nodes of any degree greater than one, while SwapOperandsMutation remains enabled only when the tree contains a binary node.

A composable plugin interface

Plugins are the new extension point for the search loop (#645, #663). A plugin is a small struct subtyping AbstractPlugin: an immutable configuration paired with mutable runtime state created by init_plugin_state. Hooks cover lifecycle events (on_search_start!, on_generation_end!, on_cycle_start!/on_cycle_end!, on_mutation_end!, on_search_end!), selection and acceptance biases (tournament_cost_multiplier, mutation_acceptance_multiplier), mutation conditioning (condition_mutation!), and population seeding (init_member). Plugins can also contribute weighted mutation and crossover defaults through plugin_mutations and plugin_crossovers, which explicit mutations=/crossovers= entries override. Multiple plugins compose in tuple order, and extending the search no longer means forking the package.

using SymbolicRegression
using SymbolicRegression: AbstractPlugin, AbstractMutation, MutationEvent

struct MutationCounterPlugin <: AbstractPlugin end

mutable struct MutationCounterState
    accepted::Int
    rejected::Int
end

function SymbolicRegression.init_plugin_state(::MutationCounterPlugin, options, dataset)
    return MutationCounterState(0, 0)
end

function SymbolicRegression.on_mutation_end!(
    state::MutationCounterState, ::MutationCounterPlugin, ::AbstractMutation,
    event::MutationEvent, dataset, options,
)
    event.accepted ? (state.accepted += 1) : (state.rejected += 1)
    return nothing
end

model = SRRegressor(;
    binary_operators=[+, -, *, /],
    plugins=(MutationCounterPlugin(),),
)

The default set contains SimulatedAnnealingPlugin, AdaptiveParsimonyPlugin, and AdaptiveMutationWeightsPlugin. Annealing carries the previous temperature schedule; adaptive parsimony biases selection away from over-represented complexities; adaptive mutation weights learn multiplicative factors from observed improvements (see Changed defaults). MutationBurstPlugin is opt-in. Pass plugins=(...) to add or replace by type, and default_plugins=() to run the core loop without automatic plugins. The interface is experimental and may change in future releases.

First-class mutations and crossovers

Mutations are now types passed with weights, and each built-in carries its own hyperparameters (#610). An entry replaces the default weight for that mutation type; anything you leave out keeps its default, and default_mutations=() removes every automatic entry.

model = SRRegressor(
    binary_operators=[+, -, *],
    mutations=[
        OptimizeMutation() => 0.1,
        ConstantMutation(perturbation_factor=0.1) => 0.5,
    ],
)

The fifteen built-ins are ConstantMutation, OperatorMutation, FeatureMutation, SwapOperandsMutation, RotateTreeMutation, AddNodeMutation, InsertNodeMutation, DeleteNodeMutation, SimplifyMutation, RandomizeMutation, OptimizeMutation, BacksolveMutation, DoNothingMutation, and the graph-only pair FormConnectionMutation and BreakConnectionMutation. FeatureMutation makes rewiring a leaf to a different input column its own weighted move (#475), which previously happened only as a side effect of delete-then-add.

Writing your own follows normal Julia dispatch. Subtype AbstractMutation, extend SymbolicRegression.mutate!, and return a MutationResult:

using SymbolicRegression
using SymbolicRegression: AbstractMutation, MutationResult
using DynamicExpressions: get_contents, AbstractExpression

struct PruneMutation <: AbstractMutation end

function SymbolicRegression.mutate!(
    new_tree::N, parent_member::P, ::PruneMutation, options; nfeatures, kws...
) where {N<:AbstractExpression,P}
    tree = get_contents(new_tree)
    nodes = filter(n -> n.degree > 0, collect(tree))
    if !isempty(nodes)
        target = rand(nodes)
        target.degree = 0
        target.feature = rand(1:nfeatures)
    end
    return MutationResult{N,P}(; tree=new_tree)
end

model = SRRegressor(
    binary_operators=[+, -, *, /],
    mutations=[PruneMutation() => 0.1],
)

Crossovers mirror the design (#664, #666): subtype AbstractCrossover, extend SymbolicRegression.crossover, return a CrossoverResult with two children, and pass it as crossovers=[MyCrossover() => weight]. The default SubtreeCrossover stays in place unless you remove it with default_crossovers=(). When children violate constraints the engine retries the sampled crossover and passes a 1-based attempt keyword, so a crossover backed by an expensive external model can return copies of the parents on retries.

The experimental BacksolveMutation

BacksolveMutation (#573, thanks @ayagh19) targets failures random perturbation handles poorly, such as a correct outer form around a wrong inner argument. It walks up the tree, inverting each operator on the path to the root to compute what a subtree should have produced, then fits a sparse replacement by greedy forward selection over a library of the best subtrees currently in the population, constrained by the remaining complexity budget. It is off by default (BacksolveMutation() => 0.0) and flagged experimental.

Seed the search with guesses

Give equation_search any guess for the final expressions, and it mixes those guesses into the populations throughout the search (#469, #500). Each guess is parsed with your configured operators. When should_optimize_constants=true, its constants are optimized before it joins a population. fraction_replaced_guesses controls the fraction mixed in at the end of each cycle, so guesses continue contributing throughout the run even when their initial constants are inaccurate.

using SymbolicRegression
using SymbolicRegression: machine, fit!, report

X = randn(Float32, 6, 2048)
y = @. sin(X[1, :] * X[2, :] + 0.1f0) + cos(X[3, :]) * X[4, :] +
       X[5, :] / (X[6, :] * X[6, :] + 1)

model = SRRegressor(
    binary_operators=[+, -, *, /],
    unary_operators=[sin, cos],
    guesses=["sin(x1 * x2) + cos(x3) * x4 + x5 / (x6 * x6 + 0.9)"],
    niterations=35,
)
mach = machine(model, X', y)
fit!(mach)

For multi-output searches pass a vector of vectors, one inner vector per output. With TemplateExpressionSpec, guesses are named tuples keyed by sub-expression name, using #1, #2 as placeholders for the arguments, such as (; f="cos(#1) + 0.1", g="sin(#2) - 0.9"). Guess constants evaluate inside a generated module, so guesses naming custom operators resolve correctly (#705), and overly complex guesses produce a warning rather than silently dominating a population.

A reusable evaluation arena

Evaluation buffers are allocated once in a contiguous arena and reused across mutation, crossover, loss evaluation, constant optimization, and template inner calls (#654, #668; DynamicExpressions #180, #186). On SymbolicRegression.jl's own 8-thread benchmark suite, as reported in those pull requests, a full search dropped from 9.541 s to 5.880 s median and allocated bytes fell from 59.10 GB to 10.71 GB, with the hall of fame byte-identical across the change. Treat these as the backend's measured workload, not a universal multiplier.

Ordinary users configure nothing. At the library level, EvalOptions became EvalContext with caller-owned arena lifetimes (DynamicExpressions #187, #192); the old binding remains as a deprecated alias, and evaluation entry points now reject unknown keywords instead of ignoring them (#670). Separately, precompilation uses single-output searches for Float32 and, by default, Float64. Set the precompile_float64 preference to false to omit the Float64 workload (#642).

Automatic batching and adaptive mutation defaults

With batching=:auto and batch_size=nothing now the defaults, searches over large datasets minibatch without configuration (#676). Datasets of 1000 rows or fewer use full data; above that the chosen batch size is 128 rows below 5000, 256 below 50000, and 512 otherwise, capped by an explicit batch_size. Minibatches guide the inner evolution, while hall-of-fame members are reevaluated on the full dataset before they are returned. Restore full-data evolution with batching=false, batch_size=50.

Adaptive mutation weights are on by default (#678): AdaptiveMutationWeightsPlugin tracks attempts and strict improvements per population and applies learned multiplicative factors, regularized toward your configured weights in log space. The enabling pull request's continuous-benchmark runs showed multithreaded runtime of 12.9 s without adaptation versus 13.1 s with it, an aggregate score improvement of 0.0143, so the mechanism operates at roughly parity overhead on that workload. Disable it alone with plugins=(AdaptiveMutationWeightsPlugin(adaptation_strength=0),) or drop all automatic plugins with default_plugins=().

The standard workflow without MLJ

machine, fit!, predict, and report work whenever a Tables-compatible input is supplied, through a new SymbolicRegressionTablesExt, so fitting a regressor no longer loads MLJ or MLJBase (#680). The extension handles table detection, column names, matrix conversion, and result materialization.

using SymbolicRegression
using SymbolicRegression: machine, fit!, predict, report

X = 2randn(1000, 5)
y = @. 2*cos(X[:, 4]) + X[:, 1]^2 - 2

model = SRRegressor(
    binary_operators=[+, -, *, /],
    unary_operators=[cos],
    niterations=30,
)
mach = machine(model, X, y)
fit!(mach)
r = report(mach)
println(r.equations[r.best_idx])
yhat = predict(mach, randn(10, 5))

Existing MLJ users keep the same surface: the models remain MLJ-compatible, and the core still uses MLJModelInterface.

Expansions for custom value types

Searching over nonnumeric values is not new: the string interface arrived in v1.10 and v1.12 already shipped the generic value interface, with GenericOperatorEnum, init_value, sample_value, mutate_value, scalar-constant counting, and custom printing, as shown in examples/custom_types.jl. Likewise, D(f, i) from DynamicDiff inside @template_spec predates this release. Version 2 expands these interfaces:

  • Operators of arbitrary arity work with GenericOperatorEnum, so custom structs can flow through ternary and higher-degree nodes.
  • Template expressions accept custom value types in their bodies (#690) and as parameter vectors (#693); parameter vectors themselves became generic and optimizable alongside the expression constants in #644, generalizing what TemplateStructure(; num_parameters=...) already offered in v1.
  • Discrete custom-value mutation works again: a regression broke mutate_value moves for noncontinuous types, fixed in #687.
  • DynamicDiff compatibility moved to 0.3, whose derivative operator supports expressions containing n-ary operator nodes (DynamicDiff #4).

Changed defaults

Search dynamics differ from v1.13 even when your code runs unchanged.

setting v1.13 v2.0 note
batching false :auto engaged above 1000 rows; hall-of-fame members are reevaluated on the full dataset before return (#676)
batch_size 50 nothing full data up to 1000 rows, then 128 / 256 / 512 by size (#676)
crossover_probability 0.0259 0.20 about eight times more recombination; chosen by a 560-search factorial ablation showing +2.24% aggregate held-out Pareto NMSE and a 420-search sweep in which 0.20 was the only setting that helped, both reported in #643
adaptive mutation weights off on AdaptiveMutationWeightsPlugin in the default set (#678)

Constant optimization can now escape zero-valued constants (#637), which changes optimization trajectories as well. To recover static v1-style weights, choose the plugin set explicitly:

Options(;
    default_plugins=(SimulatedAnnealingPlugin(; alpha=3.17), AdaptiveParsimonyPlugin()),
    batching=false,
    batch_size=50,
    crossover_probability=0.0259,
)

Migration notes

Most renamed keywords continue to work through warning shims. The remaining migration points are below.

Renames with working, warning shims:

  • eval_options= becomes eval_context= in evaluation entry points; EvalOptions remains as a deprecated alias for EvalContext.
  • use_recorder/recorder_file become use_tracing/tracing_file, writing versioned JSONL records (#651).
  • PopMember.score becomes PopMember.cost.
  • camelCase keywords such as mutationWeights, useFrequency, and shouldOptimizeConstants convert automatically to snake_case.

Removals and signature changes:

  • ParametricExpression, ParametricNode, and ParametricExpressionSpec are gone. Parameterized template expressions cover the same ground generically: @template_spec(expressions=(f,), parameters=(p=2,)) declares a parameter vector optimized with the constants (#656, #644).
  • Options, SearchState, and TemplateExpressionSpec gained type parameters, changing their concrete type arity, and SearchState replaces all_running_search_statistics with plugin_states.
  • Node{T} is now Node{T,D} with the maximum arity as the second parameter (DynamicExpressions #127).
  • The internal delete_random_op! helper gained an n-ary signature. _random_op was removed; custom mutations should use append_random_op, insert_random_op, or prepend_random_op as appropriate.

Other changes

  • Tracing is centralized: Options(; use_tracing=true, tracing_file="run.jsonl") writes a versioned JSONL record stream covering population members, mutation events, crossover details, costs, and parent references (#651). Disabled tracing is designed to be zero-allocation, and memory scales with in-flight records rather than the whole search history.
  • Reliability fixes: mismatched X/y sample counts error immediately with a clear message (#660); multiprocessing teardown no longer hangs (#641); members violating constraints are never stored; poisson_sample handles lambda=0; gradient evaluation works with SubArray inputs (#566); stdin quit monitoring is non-blocking (#562); expression-level losses skip simplification (#674); and discrete custom-value mutation works again (#687).
  • Diagnostics: worker_timeout now controls connection startup by setting JULIA_WORKER_TIMEOUT while workers are created (commits 5b4a712 and f45e146); template expression mistakes raise specific errors; hall-of-fame CSV output escapes embedded quotes; iteration counters display consistently with progress=false.
  • The simulated annealing temperature schedule survived its port to SimulatedAnnealingPlugin bit for bit, verified by identical hall-of-fame hashes (#652).

Dependency versions

SymbolicRegression.jl 2.0.0 (tag v2.0.0) requires Julia 1.10 or later and uses:

  • DynamicExpressions.jl ~2.10 (up from ~1.10.1/~1.11).
  • DynamicDiff.jl 0.3 (up from 0.2).
  • SymbolicUtils.jl 4, as an optional extension for symbolic conversion.
  • Optional extension packages for autodiff backends: Mooncake 0.4.137/0.5 and Enzyme 0.12/0.13, selected with autodiff_backend=:Mooncake or :Enzyme alongside the existing Zygote route.
  • Tables.jl as an optional extension backing the MLJ-free workflow, and JSON3.jl for trace consumers.

Docs: ai.damtp.cam.ac.uk/symbolicregression. Repo: github.com/astroautomata/SymbolicRegression.jl.

All pull requests

Complete pull request list (135)

New Contributors

Full Changelog: v1.13.4...v2.0.0

Contributors

wsmoses, spinnau, and 7 other contributors
Assets 2
Loading