No description, website, or topics provided.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
scripts
sql
README.md

README.md

redshift-benchmark

This project includes the data set for Redshift benchmark.

Resources

  • Test Data
    • s3://hapyrus-examples/redshift-benchmark/ad-network-examples/case-01/ad_campaigns
    • s3://hapyrus-examples/redshift-benchmark/ad-network-examples/case-01/advertisers
    • s3://hapyrus-examples/redshift-benchmark/ad-network-examples/case-01/publishers
    • s3://hapyrus-examples/redshift-benchmark/ad-network-examples/case-01/imp_logs
    • s3://hapyrus-examples/redshift-benchmark/ad-network-examples/case-01/click_logs

Requirements

  • Redshift cluster
  • Local environment
    • postgresql client

Import the data set into Redshift cluster

  • Create tables
    • run sql/create_tables_redshift.sql on your Redshift cluster
  • Copy the test data set on our s3 to your Redshift cluster(it took over 17 hours in our case)
    • edit sql/copy_all_[data-size].sql and write your own [aws-access-key-id] and [aws-secret-access-key].
    • run sql/copy_all_[data-size].sql on your Redshift cluster
  • Run the test sql
    • run sql/test-query.sql on your Redshift cluster

Hadoop + Hive

  • See sql/create_tables_hadoop_hive.sql to create Hive tables.
  • Run the test sql
    • see sql/test-query.sql

License

This project is liscensed under the Apache License, Version 2.0 and powered by Hapyrus Inc.