Skip to content

Commit

Permalink
Merge 6f4fac4 into 28e58b8
Browse files Browse the repository at this point in the history
  • Loading branch information
KoteiIto committed Jan 1, 2018
2 parents 28e58b8 + 6f4fac4 commit 993d03d
Show file tree
Hide file tree
Showing 42 changed files with 4,813 additions and 1,015 deletions.
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,7 @@ jspm_packages
.npm

# Optional REPL history
.node_repl_history
.node_repl_history

.DS_Store
index.js
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@ node_js:
- '6'
- '5'
- '4'
script: 'make test-cov'
script: 'make test'
after_success: 'make coveralls; make clean'
15 changes: 6 additions & 9 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,14 +1,11 @@
mocha=./node_modules/.bin/_mocha
istanbul=./node_modules/.bin/istanbul
build: clean
yarn run build

test: clean
$(mocha)

test-cov: clean
$(istanbul) cover $(mocha) -- -R spec test/*
test:
yarn run test

coveralls:
cat ./coverage/lcov.info | ./node_modules/coveralls/bin/coveralls.js;
yarn coveralls

clean:
rm -fr coverage
rm -rf build | rm -rf coverage
115 changes: 85 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,42 @@
[![Build Status](https://travis-ci.org/KoteiIto/node-athena.svg?branch=master)](https://travis-ci.org/KoteiIto/node-athena)
[![Coverage Status](https://coveralls.io/repos/github/KoteiIto/node-athena/badge.svg?branch=master)](https://coveralls.io/github/KoteiIto/node-athena?branch=master)

athena-client - a nodejs simple aws athena client
athena-client - a simple aws athena client for nodejs and typescript
===========================
This is version 2.x document. 1.x document is [here](https://github.com/KoteiIto/node-athena/tree/1.x)

Install with:

npm install athena-client

## Usage Example

### Create Client
```js
var awsConfig = {
region: 'xxxx',
}
var clientConfig = {
bucketUri: 's3://xxxx'
}

var awsConfig = {
region: 'xxxx',
}

var athena = require("athena-client")
var client = athena.createClient(clientConfig, awsConfig)

```

### Receive result by Callback
```js
client.execute('SELECT 1', function(err, data) {
if (err) {
return console.error(err)
}
console.log(data)
})

// You can also execute query with promises
```

### Receive result by Promise
```js
client.execute('SELECT 1').toPromise()
.then(function(data) {
console.log(data)
Expand All @@ -36,49 +45,95 @@ client.execute('SELECT 1').toPromise()
})
```

### Receive result by Stream
```js
var stream = client.execute('SELECT 1').toStream()
stream.on('data', (record) => {
console.log(record)
})
stream.on('query_end', (queryExecution) => {
console.log(queryExecution)
})
stream.on('end', () => {
console.log('end')
})
stream.on('error', (e) => {
console.error(e)
})
```

# API
### athena = require("athena-client")
This module exposes the `createClient` method, which execute query to AWS Athena

### client = athena.createClient([_clientConfig_], [_awsConfig_])
Returns a client instance attached to the account specified by the given clientConfig and awsConfig .

Returns a client instance attached to the account specified by the given clientConfig and awsConfig.

#### `clientConfig` object properties
| Property | Default | Description |
|-----------|-----------|-------------|
| bucketUri | __Required__ | URI of S3 bucket for saving a query results file(*.csv) and a metadata file (*.csv.metadata) |
| pollingInterval | 1000 | Interval of polling sql results (ms) |
| queryTimeout | 0 | Timeout of query execution. `0` is no timeout |
| concurrentExecMax | 5 | The number of cuncurrent execution of query max. it should be set `smaller than AWS Service limit`(default is 5) |
| concurrentExecMax | 5 | The number of cuncurrent execution of query max. It should be set `smaller than AWS Service limit`(default is 5) |
| maxBufferSize | '128M' | Maximum buffer when retrieving query results |

#### `awsConfig` object properties
| Property | Default | Description |
|-----------|-----------|-------------|
| region | __Required__ | Your Athena and S3 region |
| accessKeyId | undifined | Your IAM accessKeyId. This is optional |
| secretAccessKey | undifined | Your IAM secretAccessKey. This is optional |

The awsConfig can be specified as an object with `region` and `accessKeyId` and `secretAccessKey` members such as the following:
### client.execute([_query_], [_callback_])
It will return the following result.
If you want to know more about params of `queryExecution`, please refer to the aws-sdk [document](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Athena.html#getQueryExecution-property)

```javascript
var awsConfig = {
region: 'xxxx',
accessKeyId: 'xxxx', // Optional
secretAccessKey: 'xxxx', // Optional
```json
{
"records": [
{"_col0:": "1"}
],
"queryExecution": {
"Query": "SELECT 1",
"QueryExecutionId": "55571bb9-8e4e-4274-90b7-8cffe4539c3c",
"ResultConfiguration": {
"OutputLocation": "s3://bucket/55571bb9-8e4e-4274-90b7-8cffe4539c3c"
},
"Statistics": {
"DataScannedInBytes": 0,
"EngineExecutionTimeInMillis": 137
},
"Status": {
"CompletionDateTime": "2017-12-31T16:03:53.493Z",
"State": "SUCCEEDED",
"SubmissionDateTime": "2017-12-31T16:03:53.209Z"
}
}
}
```

### client.execute([_query_], [_callback_])
Returns query result.
### client.execute([_query_]).toPromise()
It will return promise object to get result.

```javascript
client.execute('SELECT 1', function(err, data) {
if (err) {
return console.error(err)
}
console.log(data)
### client.execute([_query_]).toStream()
It will return stream object to get result. If your query results are very `large`, we recommend using this stream.

```js
// Get record one by one
stream.on('data', (record) => {
console.log(record) // {"col1": "val1", "col2": "val2"}
})

client.execute('SELECT 1').toPromise()
.then(function(data) {
console.log(data)
}).catch(function(err) {
console.error(err)
// When query succeed, this event will emit.
stream.on('query_end', (queryExecution) => {
console.log(queryExecution) // {"QueryExecutionId": "", ...}
})

```
stream.on('end', () => {
console.log('end')
})
stream.on('error', (e) => {
console.error(e)
})
```
23 changes: 23 additions & 0 deletions build/index.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions build/index.js.map

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

144 changes: 144 additions & 0 deletions build/lib/client.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions build/lib/client.js.map

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 993d03d

Please sign in to comment.