Skip to content

Commit

Permalink
add until; update header; linter hot-fix
Browse files Browse the repository at this point in the history
  • Loading branch information
zhenik committed Sep 11, 2020
1 parent 09e586f commit c852db3
Show file tree
Hide file tree
Showing 4 changed files with 33 additions and 10 deletions.
3 changes: 2 additions & 1 deletion .github/workflows/on_pr_push_master.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ jobs:
steps:
- uses: actions/checkout@v2
- name: Super-Linter
uses: github/super-linter@v3.9.4
# todo: use github/super-linter when will be resolved https://github.com/github/super-linter/issues/708
uses: Neha-Sinha2305/super-linter@master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VALIDATE_ANSIBLE: true
Expand Down
4 changes: 2 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@

### Added

- Data examples & create tables
- Documentation
- Data examples & create tables #5 #4
- Documentation #3 #8
- Fixate linter version #10

## [0.0.1]
Expand Down
27 changes: 20 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,25 @@
<!-- markdownlint-disable MD041 -->
<p align="center">
<h2 align="center">Terraform-nomad-presto</h2>
<p align="center">Terraform module with example</p>
<a href="https://github.com/fredrikhgrelland/vagrant-hashistack/releases">
<img alt="Releases" src="https://img.shields.io/badge/dynamic/json?label=with%20vagrant-hashistack&query=%24.current_version.version&url=https%3A%2F%2Fapp.vagrantup.com%2Fapi%2Fv1%2Fbox%2Ffredrikhgrelland%2Fhashistack"/>
</a>
<h2 align="center">Terraform-nomad-presto</h2>
</p>
<p align="center">
<a href="https://github.com/fredrikhgrelland/vagrant-hashistack-template" alt="Built on">
<img src="https://img.shields.io/badge/Built%20from%20template-Vagrant--hashistack--template-blue?style=for-the-badge&logo=github"/>
</a>
<p align="center">
<a href="https://github.com/fredrikhgrelland/vagrant-hashistack" alt="Built on">
<img src="https://img.shields.io/badge/Powered%20by%20-Vagrant--hashistack-orange?style=for-the-badge&logo=vagrant"/>
</a>
</p>
</p>

---

Module contains a nomad job [./conf/nomad/presto.hc](./conf/nomad/presto.hcl) with [presto sql server](https://github.com/prestosql/presto).

#
Additional information:
- [consul-connect](https://www.consul.io/docs/connect) integration
- [nomad docker driver](https://www.nomadproject.io/docs/drivers/docker.html)

## Contents
0. [Prerequisites](#prerequisites)
Expand Down Expand Up @@ -194,7 +207,7 @@ presto --server localhost:8080 --catalog hive --schema default --user presto --f

## License

________
---

## References
- [Blog post](https://towardsdatascience.com/load-and-query-csv-file-in-s3-with-presto-b0d50bc773c9)
Expand Down
9 changes: 9 additions & 0 deletions dev/ansible/05_presto_create_tables.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,16 +37,25 @@
- name: Create CSV table
shell: docker run --network host -v "/vagrant/example/resources/query/csv_create_table.sql:/csv_create_table.sql" fredrikhgrelland/presto-cli ./presto --server localhost:8888 --catalog hive --schema default --user presto --file /csv_create_table.sql
register: docker_output_cmd_csv
retries: 5
delay: 1
until: docker_output_cmd_csv.rc == 0
tags: example-upload

- name: Create JSON table
shell: docker run --network host -v "/vagrant/example/resources/query/json_create_table.sql:/json_create_table.sql" fredrikhgrelland/presto-cli ./presto --server localhost:8888 --catalog hive --schema default --user presto --file /json_create_table.sql
register: docker_output_cmd_json
retries: 5
delay: 1
until: docker_output_cmd_json.rc == 0
tags: example-upload

- name: Create AVRO table
shell: docker run --network host -v "/vagrant/example/resources/query/avro_tweets_create_table.sql:/avro_tweets_create_table.sql" fredrikhgrelland/presto-cli ./presto --server localhost:8888 --catalog hive --schema default --user presto --file /avro_tweets_create_table.sql
register: docker_output_cmd_avro
retries: 5
delay: 1
until: docker_output_cmd_avro.rc == 0
tags: example-upload

- name: Verify CSV table - available 150 records
Expand Down

0 comments on commit c852db3

Please sign in to comment.