substitutions for powers with new symbolic variables using Symbolics.jl

[xiang-yu]

I have a vector "g" with high order polynomial elements x^alpha. I want to replace the high order polynomials x^alpha with first order variables "phi".
Here is my Julia code using Symbolics.jl,

phi = Symbolics.variables(:phi, 1:3)

g = Symbolics.variables(:g, 1:3)
g[1] = a * x^2 + b * y^2 + d * xy + 1
g[2] = b * x^2 + a * y^2 + d * xy + 1
g[3] = 0

m = Symbolics.variables(:m, 1:3)
m[1] = x^2
m[2] = y^2
m[3] = xy

expr = Symbolics.variables(:x, 1:3)
for i = 1:3
expr[i] = substitute(g[i], Dict(m[i]=>phi[i]))
end

This returns,
julia> expr
3-element Vector{Num}:
g₁
g₂
g₃

This is not right because it should be

aphi1 + bphi2 + d phi3 +1
bphi1 + a*phi2 + d *phi3+1
0

[bowenszhu]

using Symbolics
@variables x y xy a b d
ϕ = Symbolics.variables(:ϕ, 1:3)
g = [a * x^2 + b * y^2 + d * xy + 1,
    b * x^2 + a * y^2 + d * xy + 1,
    0]
m = [x^2,
    y^2,
    xy]
expr = substitute(g, zip(m, ϕ)) # or substitute.(g, (Dict(zip(m, ϕ)),))

3-element Vector{Num}:
 1 + a*ϕ₁ + b*ϕ₂ + d*ϕ₃
 1 + b*ϕ₁ + a*ϕ₂ + d*ϕ₃
                      0

[xiang-yu]

Thanks a lot. This solves my sample problem.
Here is the real problem,

@variables x y z a b d

g = [a * x^2 + b * z^2 + d * xz + x + y + z + 1, a * x^2 + b * z^2 + d * xz + x + y + z + 1, a * x^2 + b * z^2 + d * xz + x + y + z + 1, a * x^2 + b * z^2 + d * xz + x + y + z + 1, a * x^2 + b * z^2 + d * x*z + x + y + z + 1, 0]

ϕ = Symbolics.variables(:ϕ, 1:6)

m = [x*z, x^2, z^2, x, y, z]

expr = substitute(g, zip(m, ϕ))
6-element Vector{Num}:
1 + ϕ₄ + ϕ₅ + ϕ₆ + aϕ₂ + bϕ₃ + dϕ₄ϕ₆
1 + ϕ₄ + ϕ₅ + ϕ₆ + aϕ₂ + bϕ₃ + dϕ₄ϕ₆
1 + ϕ₄ + ϕ₅ + ϕ₆ + aϕ₂ + bϕ₃ + dϕ₄ϕ₆
1 + ϕ₄ + ϕ₅ + ϕ₆ + aϕ₂ + bϕ₃ + dϕ₄ϕ₆
1 + ϕ₄ + ϕ₅ + ϕ₆ + aϕ₂ + bϕ₃ + dϕ₄ϕ₆
0

The idea is to replace the nonlinear terms like x^2, z^2 and x*z by ϕi,
so the returned expr should be

1 + ϕ₄ + ϕ₅ + ϕ₆ + aϕ₂ + bϕ₃ + dϕ₁
1 + ϕ₄ + ϕ₅ + ϕ₆ + aϕ₂ + bϕ₃ + dϕ₁
1 + ϕ₄ + ϕ₅ + ϕ₆ + aϕ₂ + bϕ₃ + dϕ₁
1 + ϕ₄ + ϕ₅ + ϕ₆ + aϕ₂ + bϕ₃ + dϕ₁
1 + ϕ₄ + ϕ₅ + ϕ₆ + aϕ₂ + bϕ₃ + dϕ₁
0
It seems that the code is confused with the xz term. Instead of substituting it with ϕ₁, the code substitutes it by ϕ₄*ϕ₆.
Any idea about this? Any guidance will be highly appreciated!

[bowenszhu]

substitute doesn't work in this case.
The expression d * x * y is represented as follows

  Mul
 / | \
d  x  z

substitute internally does pre-order traversal. It checks if the node d * x * z as a whole matches any given substitution rules. If not, it goes down to the children nodes. Therefore, the following doesn't work as you might have expected.

using Symbolics
@variables d x z a
res = substitute(d * x * z, Dict(x * z => a)) # returns d*x*z

[xiang-yu]

Indeed, that is precisely my problem. Is there a workaround for this?
I have quite some terms like f1f3, f2f3, ...
Sympy works, but I cannot use it because of the speed.

using SymPy
@syms x y z a
ex = xy
subs(ex, xy, a)

[bowenszhu]

There is another tool in Symbolics called term rewriter.

(@v1.8) pkg> st -m SymbolicUtils
Status `~/.julia/environments/v1.8/Manifest.toml`
  [d1185830] SymbolicUtils v0.19.11

(@v1.8) pkg> st -m Symbolics
Status `~/.julia/environments/v1.8/Manifest.toml`
  [0c5d862f] Symbolics v4.13.0

using Symbolics, SymbolicUtils.Rewriters
@syms x y z a b d
ϕ = Symbolics.variables(:ϕ, 1:6)
rules = [
    @rule x^2 => ϕ[1]
    @rule y^2 => ϕ[2]
    @acrule x * z => ϕ[3]
    @rule x => ϕ[4]
    @rule y => ϕ[5]
    @rule z => ϕ[6]
]
expr = d * x * z + 1
res = Prewalk(RestartedChain(rules))(expr)

This returns 1 + d*ϕ₃, which is fine.
But the following errors

expr = d * x * z + a * x^2 + 1
res = Prewalk(RestartedChain(rules))(expr)

It currently has some issues, one of which is

• similarterm inconsistent with Symbolic arithmetic operations SymbolicUtils.jl#462.

I am afraid that we'll have to wait for that getting fixed.

[xiang-yu]

Thanks. Hope JuliaSymbolics/SymbolicUtils.jl#462 can be fixed soon. But in a more general purpose, any chance x*y can still be substituted using the generic "substitute" of Symbolics.jl because this could be very useful?

I tried SymEngine.jl and it works, but I still prefer Symbolics because it is built using Julia from the ground.
julia> using SymEngine
@vars x y a
(x, y, a)
julia> ex = xy
xy
julia> subs(ex, x*y, a)
a

I tried to tackle this issue with a combination of Sympy.jl and Symbolics.jl and it returned the same results,

julia> using Symbolics
julia> using SymPy
julia> @variables d, x, z, y, a
julia> subs(d * x * z * y, x * z * y => a)
dxy*z

Now I understand that the issue stems from how "x*y" is interpreted by Symbolics instead of "substitute".

[xiang-yu]

@bowenszhu Has issue JuliaSymbolics/SymbolicUtils.jl#462 being resolved? I had a quick look at it and found it was closed.

[bowenszhu]

Yeah. But we need to wait for a new release of SymbolicUtils.jl.

[xiang-yu]

Thanks. Do I understand correctly that the substitution function should work for "*" operations after this release?
subs(d * x * z * y, x * z * y => a)

[bowenszhu]

I think so. It is term rewriter, not substitute.

using Symbolics, SymbolicUtils.Rewriters
@syms x y z a b d
ϕ = Symbolics.variables(:ϕ, 1:6)
rules = [
    @rule x^2 => ϕ[1]
    @rule y^2 => ϕ[2]
    @acrule x * z => ϕ[3]
    @rule x => ϕ[4]
    @rule y => ϕ[5]
    @rule z => ϕ[6]
]
expr = d * x * z + a * x^2 + 1
res = Prewalk(RestartedChain(rules))(expr)

With Julia v1.8.2, Symbolics v4.13.0 and SymbolicUtils v4.19.11, the above code throws an error.
After changing SymbolicUtils to its master branch commit JuliaSymbolics/SymbolicUtils.jl@490011e, it correctly returns

1 + a*ϕ₁ + d*ϕ₃

[xiang-yu]

Thanks. I couldn't find the estimated release date at "https://github.com/JuliaSymbolics/SymbolicUtils.jl". @shashi @bowenszhu any estimated release date :) ?

I see that you need to specify different rules for x^2 and x * z using rewriter.
My ϕ and expr are two vectors and their element are in the form of x^2y or other combinations. Also, the occurrence of x^2y or x*y are random. Is there a way to substitute expr by ϕ in this case using rewrite?

[xiang-yu]

For a temporary fix, I tried the following,

using Symbolics
using SymPy: subs
@variables x, y, z, xx
z = x^2
subs(z, x^2=>xx)

It returns "x^2" instead of "xx". Any idea about this? Thanks a lot in advance!

PS: I tried to install the master version of SymbolicUtils.jl but got the following error,
pkg> add SymbolicUtils#master

Updating git-repo `https://github.com/JuliaSymbolics/SymbolicUtils.jl.git`

Resolving package versions...
ERROR: Unsatisfiable requirements detected for package SymbolicUtils [d1185830]:
SymbolicUtils [d1185830] log:
├─possible versions are: 0.20.0 or uninstalled
├─restricted to versions 0.18-0.19 by Symbolics [0c5d862f] — no versions left
│ └─Symbolics [0c5d862f] log:
│ ├─possible versions are: 4.13.0 or uninstalled
│ ├─restricted to versions * by symSubs [9a05d71f], leaving only versions 4.13.0
│ │ └─symSubs [9a05d71f] log:
│ │ ├─possible versions are: 0.1.0 or uninstalled
│ │ └─symSubs [9a05d71f] is fixed to version 0.1.0
│ └─Symbolics [0c5d862f] is fixed to version 4.13.0
└─SymbolicUtils [d1185830] is fixed to version 0.20.0-DEV