Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify redshift adapter to support spectrum #117

Merged

Conversation

ragi256
Copy link
Collaborator

@ragi256 ragi256 commented Mar 29, 2019

This PR is for support Redshift Spectrum. The modification is only redshift_adapter.rb

  • fetch_table_names method change to use table from pg_tables to svv_tables
  • RedshiftAdapter has instance variables for BASE TABLE, EXTERNAL TABLE, VIEW
    • to check whether a table is external table or not in fetch_rows / fetch_count
  • If importing table is external, about raw dataset

end

def spectrum?(table)
raise "@external_table_names must be defined, execute fetch_table befor spectrum?" unless instance_variable_defined?(:@external_table_names)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method depends on execution order because needs @external_table_names to check spectrum tables or not.
RedshiftAdapter is only used import tables ( synchronize_data_sources.rb).

@ragi256 ragi256 force-pushed the modify_redshift_adapter_to_support_spectrum branch from 0ad3b29 to ddd50ca Compare March 29, 2019 05:23
@ragi256 ragi256 merged commit 8bb0d73 into cookpad:master Apr 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants