Indexed.free_symbols includes itself if the base has free symbols

[terrorfisch]

Upgrading to version 1.2 breaks my code because Indexed.free_symbols now includes {self} if the base has any free symbols. Is there any reason for this? I think it's quite unintuitive as a[i] is not a free symbol if a and i are defined.

My fix is to filter the free_symbols and drop all Indexed instances. Is there a better way to get te behaviour i need?

Comits that introduced the change:
f1dcc89
327421b
@Upabjojr

[Upabjojr]

I'm sorry to hear that. The reasoning behind this change was the introduction of .expr_free_symbols. That is, .expr_free_symbols returns all free symbols that are part of the mathematical formula, while .free_symbols goes deeper.

expr.free_symbols - expr.expr_free_symbols

Maybe this could work as a stopgap measure?

The point is that Indexed is indeed a free symbol until the formula level, if you further break it down you don't have a formula anymore.

See also:
https://groups.google.com/forum/#!topic/sympy/RvPrX1BhyC8

[terrorfisch]

Thank you for the reply. I see your point.

expr_free_symbols isn't available in 1.2, right?

I use free_symbols to determine which variables I need for a numeric evaluation of expressions. I think I will use a wrapper for this.

[Upabjojr]

expr_free_symbols isn't available in 1.2, right?

It's available starting from 1.2

I use free_symbols to determine which variables I need for a numeric evaluation of expressions. I think I will use a wrapper for this.

Have you tried .atoms() or .atoms(Symbol) ?

[terrorfisch]

Wouldn't atoms would give me sum indices, integration variables and the like?

Also atoms is ~3 times slower. Doesn't matter in my case (premature optimization) and its not even an order of magnitude but I would have guessed that free_symbols is the more complicated Task.

Edit:
atoms() uses preorder_traversal which is ~3 slower than postorder_traversal

[smichr]

This behavior is also giving troubles to me in #21425:

>>> f, g = symbols('f g', cls=Function)
>>> f(x).func
f
>>> f(x).free_symbols
{x}
>>> f(x).subs(f,g)
g(x)

>>> f, g = symbols('f g', cls=IndexedBase)
>>> f[x].base
f
>>> f[x].free_symbols
{f, x, f[x]}  <---------------------------- why is `f` included
>>> f[x].subs(f,g)
g[x]

Whereas a function does not report its func as a free symbol, Indexed reports its base as a free symbol. Why is this necessary?

Ways to see symbols in an expression tree:

>>> f[x].atoms(Symbol)
{f, x}
>>> f[x].expr_free_symbols
{f[x]}
>>> f[x].free_symbols
{f, x, f[x]}

Is the reason that f is included because it refers to a data structure (as opposed to a function definition)?

[oscarbenjamin]

It's a bit fiddly here to define what we mean by a "free symbol". A Function like f that could be thought of as a symbol in many contexts and so could f(x). The fact that f is not in free symbols but subs still works is a quirk of the current implementation that f is a Basic subclass class rather than a Basic instance. Ideally it would be an instance though and functions would be first class symbolic objects that can be composed, differentiated etc. In that case f(x) would be something like:

>>> x = Symbol('x')
>>> f = FunctionSymbol('f')
>>> srepr(f(x))
Call(FunctionSymbol('f'), Symbol('x'))

The idea is that f is a function symbol that can be substituted for a known function like sin later.

Right now we couldn't return f in free symbols just because it isn't a Basic instance though.

[smichr]

I believe that allowing Indexed to list itself as a free symbol is a design mistake that should be reverted. It blurs the line between symbol and generator (a feature that I am proposing):

>>> eq=exp(x)+1
>>> generators(eq)
{exp(x)}
>>> eq.free_symbols
{x}

>>> from sympy.stats import Normal
>>> eq=Normal('X', 2, 4)
>>> generators(eq)
{X}
>>> eq.free_symbols
{X}

>>> eq=MatrixSymbol("A",2,2)[1,1]
>>> generators(eq)
{A[1, 1]}
>>> eq.free_symbols
{A}


>>> Indexed('x',0)+1
x[0] + 1
>>> eq=_
>>> generators(eq)
{x[0]}
>>> eq.free_symbols
{x, x[0]}  <---- should just be {x}

[ThePauliPrinciple]

>>> eq.free_symbols
{x, x[0]}  <---- should just be {x}

Wouldn't this imply that you can no longer substitute x[0] with e.g. x[1].

I think this would be useful, but what does "generator" mean in mathematical terms? I feel that the underlying problem with these issues is that a computer science concept is used that only maps to its mathematical counterpart in some contexts but not all (similar to classes being used as functions).

[oscarbenjamin]

Something doesn't need to be in free_symbols to be substitutable:

In [1]: e = exp(x) + log(y)

In [2]: e
Out[2]: 
 x         
ℯ  + log(y)

In [3]: e.free_symbols
Out[3]: {x, y}

In [4]: e.subs(log(y), cos(t))
Out[4]: 
 x         
ℯ  + cos(t)

[smichr]

[what does 'generator'] mean in mathematical terms

From the docstring,

    ... expressions which would appear in the expanded
    version of self as 1) the bases of powers with Integer exponents
    or else as 2) powers with non-Integer exponents (but with numerator
    free of leading Integer. Only expressions with free symbols are
    returned.  If ``all=True`` then the set will include free symbols
    of the generators, too.

I'm still thinking about what to do with (x+y)**3+(x + y)**2 + 3*x+3*y.