Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Fix fusion optimization bug that incorrectly changes operation order #2635

Merged
merged 9 commits into from
Feb 19, 2021

Conversation

icexelloss
Copy link
Contributor

@icexelloss icexelloss commented Feb 18, 2021

What change is proposed

Example code

t = ibis.table(ibis.schema([('col', 'float32')]), 't')
t = t[['col']]
t = t[~t['col'].isnan()]
t = t.mutate(col=t['col'].cast('int32'))

Before:

ref_0
UnboundTable[table]
  name: t
  schema:
    col : float32

Selection[table]
  table:
    Table: ref_0
  selections:
    col = Cast[int32*]
      col = Column[float32*] 'col' from table
        ref_0
      to:
        int32
  predicates:
    Not[boolean*]
      IsNan[boolean*]
        col = Column[float32*] 'col' from table
          ref_0

After:

ref_0
UnboundTable[table]
  name: t
  schema:
    col : float32

ref_1
Selection[table]
  table:
    Table: ref_0
  selections:
    col = Column[float32*] 'col' from table
      ref_0
  predicates:
    Not[boolean*]
      IsNan[boolean*]
        col = Column[float32*] 'col' from table
          ref_0

Selection[table]
  table:
    Table: ref_1
  selections:
    col = Cast[int32*]
      col = Column[float32*] 'col' from table
        ref_1
      to:
        int32

Testing

I added a few tests in test_analysis.py to assert the fusion behavior. Also created a test in backends test to make sure the results are expected.

@icexelloss icexelloss changed the title BUG: Fix fusion optimization bug that incorrectedly change operation … BUG: Fix fusion optimization bug that incorrectly changes operation … Feb 18, 2021
@icexelloss icexelloss changed the title BUG: Fix fusion optimization bug that incorrectly changes operation … BUG: Fix fusion optimization bug that incorrectly changes operation order Feb 18, 2021
@jreback jreback added the expressions Issues or PRs related to the expression API label Feb 18, 2021
@jreback jreback added this to the Next release milestone Feb 18, 2021
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you also add a release note

ibis/backends/tests/test_generic.py Show resolved Hide resolved
ibis/backends/tests/test_generic.py Outdated Show resolved Hide resolved
ibis/backends/tests/test_generic.py Show resolved Hide resolved
ibis/tests/expr/test_analysis.py Show resolved Hide resolved
@jreback jreback merged commit cdd1c50 into ibis-project:master Feb 19, 2021
@jreback
Copy link
Contributor

jreback commented Feb 19, 2021

thanks @icexelloss

@icexelloss
Copy link
Contributor Author

Thanks @jreback !

@cpcloud cpcloud removed this from the Next release milestone Jan 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
expressions Issues or PRs related to the expression API
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants