-
Notifications
You must be signed in to change notification settings - Fork 18
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
13 changed files
with
450 additions
and
37 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
env: | ||
- VERSION=1.1.54276 | ||
|
||
language: python | ||
python: | ||
- "3.5" | ||
- "3.6" | ||
cache: pip | ||
services: | ||
- docker | ||
before_install: | ||
- docker run -d -p 127.0.0.1:9000:9000 --name test-clickhouse-server --ulimit nofile=262144:262144 yandex/clickhouse-server:$VERSION | ||
- docker run -d --entrypoint "/bin/sh" --name test-clickhouse-client --link test-clickhouse-server:clickhouse-server yandex/clickhouse-client:$VERSION -c 'while :; do sleep 1; done' | ||
- docker ps -a | ||
# Faking clickhouse-client real comminitation with container via docker exec. | ||
- echo -e '#!/bin/bash\n\ndocker exec test-clickhouse-client clickhouse-client "$@"' | sudo tee /usr/local/bin/clickhouse-client > /dev/null | ||
- sudo chmod +x /usr/local/bin/clickhouse-client | ||
# Overriding setup.cfg. Set host=clickhouse-server | ||
- echo -e '[db]\nhost=clickhouse-server\nport=9000\ndatabase=test\nuser=default\npassword=\ncompression=lz4,lz4hc,zstd' > setup.cfg | ||
# Make host think that clickhouse-server is localhost | ||
- echo '127.0.0.1 clickhouse-server' | sudo tee /etc/hosts > /dev/null | ||
install: | ||
pip install flake8 flake8-print coveralls | ||
before_script: | ||
flake8 . | ||
script: | ||
- coverage run --source=aioch setup.py test | ||
after_success: | ||
- coveralls |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,8 @@ | ||
# Changelog | ||
|
||
## 0.0.1 | ||
## [Unreleased] | ||
|
||
## 0.0.1 - 2017-09-27 | ||
### Added | ||
- Init version. | ||
- `execute` / `execute_with_progress`wrappers. | ||
- `loop`, `executor` client params. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,73 @@ | ||
# aioch | ||
aioch - is a library for accessing a ClickHouse database over native interface from the asyncio | ||
**aioch** is a library for accessing a ClickHouse database over native interface from the asyncio. | ||
It wraps features of [clickhouse-driver](https://github.com/mymarilyn/clickhouse-driver) for asynchronous usage. | ||
|
||
[![Coverage Status](https://coveralls.io/repos/github/mymarilyn/aioch/badge.svg?branch=master)](https://coveralls.io/github/mymarilyn/aioch?branch=master) | ||
[![Build Status](https://travis-ci.org/mymarilyn/aioch.svg?branch=master)](https://travis-ci.org/mymarilyn/aioch) | ||
|
||
|
||
## Installation | ||
|
||
The package can be installed using `pip`: | ||
|
||
```bash | ||
pip install aioch | ||
``` | ||
|
||
To install from source: | ||
|
||
```bash | ||
git clone https://github.com/mymarilyn/aioch | ||
cd aioch | ||
python setup.py install | ||
``` | ||
|
||
## Usage | ||
```python | ||
from datetime import datetime | ||
|
||
import asyncio | ||
from aioch import Client | ||
|
||
|
||
async def exec_progress(): | ||
client = Client('localhost') | ||
|
||
progress = await client.execute_with_progress('LONG AND COMPLICATED QUERY') | ||
timeout = 20 | ||
started_at = datetime.now() | ||
|
||
async for num_rows, total_rows in progress: | ||
done = num_rows / total_rows if total_rows else total_rows | ||
now = datetime.now() | ||
# Cancel query if it takes more than 20 seconds to process 50% of rows. | ||
if (now - started_at).total_seconds() > timeout and done < 0.5: | ||
await client.cancel() | ||
break | ||
else: | ||
rv = await progress.get_result() | ||
print(rv) | ||
|
||
|
||
async def exec_no_progress(): | ||
client = Client('localhost') | ||
rv = await client.execute('LONG AND COMPLICATED QUERY') | ||
print(rv) | ||
|
||
|
||
loop = asyncio.get_event_loop() | ||
loop.run_until_complete(asyncio.wait([exec_progress(), exec_no_progress()])) | ||
``` | ||
|
||
For more information see **clickhouse-driver** usage examples. | ||
|
||
## Parameters | ||
|
||
* `executor` - instance of custom Executor, if not supplied default executor will be used | ||
* `loop` - asyncio compatible event loop | ||
|
||
Other parameters are passing to wrapped clickhouse-driver's Client. | ||
|
||
## License | ||
|
||
aioch is distributed under the [MIT license](http://www.opensource.org/licenses/mit-license.php). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
|
||
from aioch.client import Client | ||
|
||
|
||
VERSION = (0, 0, 1) | ||
__version__ = '.'.join(str(x) for x in VERSION) | ||
|
||
__all__ = ['Client'] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
[db] | ||
host=localhost | ||
port=9000 | ||
database=test | ||
user=default | ||
password= |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,90 @@ | ||
from codecs import open | ||
import os | ||
import re | ||
import sys | ||
from setuptools import setup | ||
|
||
|
||
PY_VER = sys.version_info | ||
|
||
|
||
if PY_VER < (3, 5): | ||
raise RuntimeError("aioch doesn't suppport Python earlier than 3.5") | ||
|
||
|
||
here = os.path.abspath(os.path.dirname(__file__)) | ||
|
||
|
||
def read_version(): | ||
regexp = re.compile('^VERSION\W*=\W*\(([^\(\)]*)\)') | ||
init_py = os.path.join(here, 'aioch', '__init__.py') | ||
with open(init_py) as f: | ||
for line in f: | ||
match = regexp.match(line) | ||
if match is not None: | ||
return match.group(1).replace(', ', '.') | ||
else: | ||
raise RuntimeError('Cannot find version in aioch/__init__.py') | ||
|
||
|
||
with open(os.path.join(here, 'README.md'), encoding='utf-8') as f: | ||
long_description = f.read() | ||
|
||
|
||
setup( | ||
name='aioch', | ||
version=read_version(), | ||
|
||
description=( | ||
'Library for accessing a ClickHouse database over native interface ' | ||
'from the asyncio' | ||
), | ||
long_description=long_description, | ||
|
||
url='https://github.com/mymarilyn/aioch', | ||
|
||
author='Konstantin Lebedev', | ||
author_email='kostyan.lebedev@gmail.com', | ||
|
||
license='MIT', | ||
|
||
classifiers=[ | ||
'Development Status :: 4 - Beta', | ||
|
||
|
||
'Environment :: Console', | ||
|
||
'Intended Audience :: Developers', | ||
'Intended Audience :: Information Technology', | ||
|
||
|
||
'License :: OSI Approved :: MIT License', | ||
|
||
|
||
'Operating System :: OS Independent', | ||
|
||
|
||
'Programming Language :: SQL', | ||
'Programming Language :: Python :: 3', | ||
'Programming Language :: Python :: 3.5', | ||
'Programming Language :: Python :: 3.6', | ||
|
||
'Topic :: Database', | ||
'Topic :: Software Development', | ||
'Topic :: Software Development :: Libraries', | ||
'Topic :: Software Development :: Libraries :: Application Frameworks', | ||
'Topic :: Software Development :: Libraries :: Python Modules', | ||
'Topic :: Scientific/Engineering :: Information Analysis' | ||
], | ||
|
||
keywords='ClickHouse db database cloud analytics asyncio', | ||
|
||
packages=['aioch'], | ||
install_requires=[ | ||
'clickhouse-driver>=0.0.6' | ||
], | ||
test_suite='nose.collector', | ||
tests_require=[ | ||
'nose' | ||
], | ||
) |
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
from logging.config import dictConfig | ||
|
||
|
||
def configure(): | ||
dictConfig({ | ||
'version': 1, | ||
'disable_existing_loggers': False, | ||
'formatters': { | ||
'standard': { | ||
'format': '%(asctime)s %(levelname)-8s %(name)s: %(message)s' | ||
}, | ||
}, | ||
'handlers': { | ||
'default': { | ||
'level': 'ERROR', | ||
'formatter': 'standard', | ||
'class': 'logging.StreamHandler', | ||
}, | ||
}, | ||
'loggers': { | ||
'': { | ||
'handlers': ['default'], | ||
'level': 'ERROR', | ||
'propagate': True | ||
}, | ||
} | ||
}) |
Oops, something went wrong.