Added FULL OUTER JOIN through Join(full=True)

[Stiivi]

I've added support for FULL OUTER JOIN through join argument full (as suggested in this post)

FULL OUTER JOIN, besides other uses, is very useful in ETLs for doing transformations on top of table differences. Using two outer joins, as suggested in the above mentioned post, unnecessarily overcomplicates statements.

[Stiivi]

Short script to demonstrate it:

import sqlalchemy as sa

engine = sa.create_engine("postgresql://localhost/data")
md = sa.MetaData(engine)

left = sa.Table("left", md,
                sa.Column("id", sa.Integer))
right = sa.Table("right", md,
                sa.Column("id", sa.Integer))

md.create_all()

insert = left.insert()
for i in range(1, 6):
    engine.execute(insert.values([i]))

insert = right.insert()
for i in range(4, 10):
    engine.execute(insert.values([i]))


cond = left.c.id == right.c.id
select = sa.select([left.c.id, right.c.id],
                   from_obj=left.join(right, onclause=cond, full=True))

result = engine.execute(select)

for row in result:
    print row

Output:

(1, None)
(2, None)
(3, None)
(4, 4)
(5, 5)
(None, 6)
(None, 7)
(None, 8)
(None, 9)

[zzzeek]

take a look in some of the dialects, particularly mysql/base.py, which has its own routine here; need to upgrade that also.

[Stiivi]

Updated.

[zzzeek]

it's good, will need tests in test/sql/test_compiler.py at least

[seth-p]

Shouldn't this say instead of LEFT OUTER JOIN rather than instead of JOIN?

[Stiivi]

With current implementation the full precedes isouter therefore if full == True then there will be FULL OUTER JOIN instead of any other kind of join, regardless of what the value of isouter is. The assumption here is: "full join" implies "outer join".

It would be more appropriate to have it a tri-state argument, however that would break backward compatibility. Another maybe correct way would be to set isouter to True everytime full is set to True, however that would unnecessarily increase complexity.

[seth-p]

Right. My comment isn't about the logic, just the documentation of the full parameter. All I mean to say is that if full == False here, it will render LEFT OUTER JOIN (rather than just JOIN, as implied by the comment), since it's constructing a Join with isouter=True. So I would think the parameter documentation should be:

        :param full: if True, render a FULL OUTER JOIN, instead of LEFT OUTER JOIN.

[Stiivi]

Makes sense, updated the docs.

[zzzeek]

just a reminder that this still needs some test coverage somewhere.

[Stiivi]

Which critical tests are missing for this pull requests to be merged?

[zzzeek]

our core tests for "compiler" functions are unfortunately still pretty crufty in their format, nevertheless having something in test/sql/test_compiler.py would be the beginning of it. There's a ball-of-mud method right now called test_joins(), I'd leave that but for the moment probably add a test_full_outer_join() method below. Then there's some custom stuff for MySQL here, so that indicates some tests in test/dialect/mysql/test_compiler.py as well (also an older crufty suite at the moment), I'd build into the suite called SQLTest for now.

[Stiivi]

Added the compiler test, but can't add the MySQL test, since I don't have MySQL instance running.

[zzzeek]

OK github is showing hundreds of subsequent commits, maybe if you rebase this it will clean up b.c. i can't even view the diff online at the moment.

mysql-specific tests can run without mysql installed in test/dialect/mysql/test_compiler.py.

[Stiivi]

Re commits: This happened after rebase with master. Anyone here with better experience with Git to tell what might have gone wrong?

If this can't be undone, I'll create a new branch/PR.

[Stiivi]

Replaced by cleaning-up the rebase mess in #209.