Variable-specific operators

[pw0908]

Hi SymbolicRegression developers!

I have a situation where I would like to constrain the regression in a way that certain operators cannot be used on specific variables. E.g. I have two variables, x1 and x2. On x1, I would like to give the regression to the option to use the log operator, but not on x2. Is there a way to specify this?

Thank you so much for your help!

Pierre

[MilesCranmer]

Hi @pw0908,

Just a quick note: if you are doing this for dimensional analysis checks (?), there is a built-in way to enforce that.

If you are doing this more generally, you could do this with a custom loss function. For example:

function my_loss(tree, dataset::Dataset{T,L}, options) where {T,L}
    idx_of_log = 1  # CHANGE ME
    num_bad_nodes = 0
    for node in tree  # Default iteration is depth-first traversal
        if node.degree == 1 && node.op == idx_of_log  # is log()
            for subnode in node.l  # Traverse children of this node
                if subnode.degree == 0 && !subnode.constant && subnode.feature == 2  # is x2
                    num_bad_nodes += 1
                end
            end
        end
    end
    # Note you can do the above code block with slightly fewer allocations if you do:
    #     `sum(n -> n.degree == 1 && n.op == 1 ? count(s -> s.degree == 0 && !s.constant && s.feature == 2, n.l) : 0, tree)`
    # This is because the `iterate(::Node)` allocates a stack, while functional methods like `foreach/count/map/sum` will just
    # directly traverse the tree! See https://ai.damtp.cam.ac.uk/dynamicexpressions/dev/examples/base_operations/
    # for details

    if num_bad_nodes > 0
        big_number = 100_000
        return L(big_number * num_bad_nodes)
    end
    prediction, valid = eval_tree_array(tree, dataset.X, options)
    if !valid
        return L(Inf)
    end
    y = dataset.y
    mse = sum(i -> abs2(prediction[i] - y[i]), eachindex(prediction, y)) / length(y)
    return mse
end

and then pass this as loss_function.

This basically will return a huge loss if there are any bad nodes.

The reason this is not return L(Inf) is just because it tends to be better for the evolution to have soft penalties rather than hard. That way the genetic algorithm knows a direction to travel (whereas if it had 5 instances of log(x2), it would have no clue).

Cheers,
Miles