Bug fixes for views: return views in get_table_options and improve reflection

[jklukas]

Because SQLAlchemy only has limited support for views, some methods that include table in the name are also supposed to return views.

[jklukas]

Since there's little distinction between tables and views, this PR also includes a refactor to reduce duplication of code between table-specific and view-specific methods.

[jklukas]

@graingert @bouk @cpcloud Thoughts?

[graingert]

What's this trim for?

[jklukas]

The output of pg_get_viewdef includes a semicolon at the end of the query. The SQLAlchemy CreateTable, etc. commands don't include a semicolon at the end of the output, so we need to strip that out somewhere for consistency.

This TRIM(TRAILING ';' is the solution used in psql's \d+ command to get rid of the semicolon.

[graingert]

This doesn't change functionality at all right?

[jklukas]

This doesn't change functionality at all right?

With this change, get_view_definition now doesn't include a semicolon in its output and get_table_options now returns an empty dict when the input relation is a view (previously, this raised an exception).

Both of the previous behaviors I consider bugs.

[graingert]

Can you add tests and changelog for these bugs please?

[jklukas]

I'm going to need some help if we're going to try to add tests for these. The current test structure for reflection relies on declarative_base objects, which AFAIK can't create views.

@graingert - Do you have suggestions about how we can create a view in our test models to reflect from?

[graingert]

One option is to use the redshift_engine pytest fixture and create views
on that using raw SQL. Beware this fixture is slower than
redshift_session because it generates a new database for each test
function.
On 11 Sep 2015 21:51, "Jeff Klukas" notifications@github.com wrote:

I'm going to need some help if we're going to try to add tests for these.
The current test structure for reflection relies on declarative_base
objects, which AFAIK can't create views.

@graingert https://github.com/graingert - Do you have suggestions about
how we can create a view in our test models to reflect from?

—
Reply to this email directly or view it on GitHub
#46 (comment)
.

[jklukas]

Test failure is an unexpected 500 from bigcrunch again.

[graingert]

For now you'll have to retry the build after redshift has booted up properly
On 14 Sep 2015 16:44, "Jeff Klukas" notifications@github.com wrote:

Test failure is an unexpected 500 from bigcrunch again.

—
Reply to this email directly or view it on GitHub
#46 (comment)
.

[graingert]

nice

[graingert]

@bouk @cpcloud @jklukas I'm pro merging this, once it has a changelog entry

[jklukas]

Totally forgot about the changelog. I just pushed an update that adds a note about the changes.

[graingert]

this file is auto-generated by zest.releaser do we need this extra new-line?

[jklukas]

The newline as an attempt to make the whitespacing consistent. Other sections in this file are preceeded by two blank lines. It looks like I missed adding an extra blank line beneath the 0.2.1 section while I was at it.

It doesn't matter functionally, so I'll revert the change to avoid noise in the diff.

[jklukas]

We're still getting intermittent 500 responses from bigcrunch. How well do you understand the behavior, @graingert?

[graingert]

@jklukas there's a small window between the cluster booting and cluster being available (DNS etc) where these 500s occur. I know how to fix this. Are you able to access the rebuild button on Travis?

[jklukas]

Are you able to access the rebuild button on Travis?

I have limited experience with Travis, so didn't know to look for one. I was just able to trigger a rebuild. We'll see how it does.

[graingert]

This looks great, merging. Thanks very much!