Skip to content

Commit

Permalink
Updated readme
Browse files Browse the repository at this point in the history
  • Loading branch information
roll committed Dec 18, 2019
1 parent ce966f0 commit 7c06a18
Show file tree
Hide file tree
Showing 3 changed files with 87 additions and 61 deletions.
2 changes: 2 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ list:

readme:
pip install md-toc
pip install referencer
referencer $(PACKAGE) README.md --in-place
md_toc -p README.md github --header-levels 3
sed -i '/(#tableschema-elasticsearch-py)/,+2d' README.md

Expand Down
103 changes: 59 additions & 44 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,12 @@ Generate and load ElasticSearch indexes based on [Table Schema](http://specs.fri

- [Getting Started](#getting-started)
- [Installation](#installation)
- [Examples](#examples)
- [Documentation](#documentation)
- [Storage](#storage)
- [Usage overview](#usage-overview)
- [Mappings](#mappings)
- [Custom mappings](#custom-mappings)
- [API Reference](#api-reference)
- [`Storage`](#storage)
- [Contributing](#contributing)
- [Changelog](#changelog)

Expand All @@ -37,9 +39,9 @@ The package use semantic versioning. It means that major versions could include
pip install tableschema-elasticsearch
```

### Examples
## Documentation

Code examples in this readme requires Python 3.3+ interpreter. You could see even more example in [examples](https://github.com/frictionlessdata/tableschema-spss-py/tree/master/examples) directory.
### Usage overview

```python
import elasticsearch
Expand Down Expand Up @@ -85,22 +87,6 @@ print(l[:10])

```

## Documentation

The whole public API of this package is described here and follows semantic versioning rules. Everyting outside of this readme are private API and could be changed without any notification on any new version.

### Storage

Package implements [Tabular Storage](https://github.com/frictionlessdata/tableschema-py#storage) interface (see full documentation on the link):

![Storage](https://i.imgur.com/RQgrxqp.png)

This driver provides an additional API:

#### `Storage(es=None)`

- `es (object)` - `elasticsearch.Elastisearc` instance. If not provided new one will be created.

In this driver `elasticsearch` is used as the db wrapper. We can get storage this way:

```python
Expand Down Expand Up @@ -188,50 +174,79 @@ Example:
}
```

#### Custom mappings
### Custom mappings

By providing a custom mapping generator class (via `mapping_generator_cls`), inheriting from the MappingGenerator class you should be able

## Contributing
## API Reference

The project follows the [Open Knowledge International coding standards](https://github.com/okfn/coding-standards).
### `Storage`
```python
Storage(self, es=None)
```
Elasticsearch Tabular Storage.

Recommended way to get started is to create and activate a project virtual environment.
To install package and development dependencies into active environment:
Package implements
[Tabular Storage](https://github.com/frictionlessdata/tableschema-py#storage)
interface (see full documentation on the link):

![Storage](https://i.imgur.com/RQgrxqp.png)

> Only additional API is documented
__Arguments__
- __es (object)__: ElasticSearch instance


#### `storage.create`
```python
storage.create(self, bucket, doc_types, reindex=False, always_recreate=False, mapping_generator_cls=None, index_settings=None)
```
$ make install
Create index with mapping by schema.

__Arguments__
- __bucket(str)__:
Name of index to be created
- __doc_types(list<(doc_type, descriptor)>)__:
List of tuples of doc_types and matching descriptors
- __always_recreate__:
Delete index if already exists (otherwise just update mapping)
- __reindex__:
On mapping mismath, automatically create
new index and migrate existing indexes to it
- __mapping_generator_cls__:
subclass of MappingGenerator
- __index_settings__:
settings which will be used in index creation


#### `storage.delete`
```python
storage.delete(self, bucket=None)
```
Delete index with mapping by schema.

To run tests with linting and coverage:
__Arguments__
- __bucket(str)__: Name of index to delete

```bash
$ make test
```

For linting `pylama` configured in `pylama.ini` is used. On this stage it's already
installed into your environment and could be used separately with more fine-grained control
as described in documentation - https://pylama.readthedocs.io/en/latest/.
## Contributing

For example to sort results by error type:
> The project follows the [Open Knowledge International coding standards](https://github.com/okfn/coding-standards).
Recommended way to get started is to create and activate a project virtual environment.
To install package and development dependencies into active environment:

```bash
$ pylama --sort <path>
$ make install
```

For testing `tox` configured in `tox.ini` is used.
It's already installed into your environment and could be used separately with more fine-grained control as described in documentation - https://testrun.org/tox/latest/.

For example to check subset of tests against Python 2 environment with increased verbosity.
All positional arguments and options after `--` will be passed to `py.test`:
To run tests with linting and coverage:

```bash
tox -e py27 -- -v tests/<path>
$ make test
```

Under the hood `tox` uses `pytest` configured in `pytest.ini`, `coverage`
and `mock` packages. This packages are available only in tox envionments.

## Changelog

Here described only breaking and the most important changes. The full changelog and documentation for all released versions could be found in nicely formatted [commit history](https://github.com/frictionlessdata/tableschema-elasticsearch-py/commits/master).
Expand Down
43 changes: 26 additions & 17 deletions tableschema_elasticsearch/storage.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,17 @@
class Storage(object):
"""Elasticsearch Tabular Storage.
It's an implementation of `tableschema.Storage`.
Package implements
[Tabular Storage](https://github.com/frictionlessdata/tableschema-py#storage)
interface (see full documentation on the link):
Args:
![Storage](https://i.imgur.com/RQgrxqp.png)
> Only additional API is documented
# Arguments
es (object): ElasticSearch instance
"""

# Public
Expand Down Expand Up @@ -89,17 +96,21 @@ def create(self, bucket, doc_types,
mapping_generator_cls=None, index_settings=None):
"""Create index with mapping by schema.
Parameters
----------
bucket: str
Name of index to be created
doc_types: list<(doc_type, descriptor)>
List of tuples of doc_types and matching descriptors
always_recreate: Delete index if already exists (otherwise just update mapping)
reindex: On mapping mismath, automatically create new index and migrate existing
indexes to it
mapping_generator_cls: subclass of MappingGenerator
index_settings: settings which will be used in index creation
# Arguments
bucket(str):
Name of index to be created
doc_types(list<(doc_type, descriptor)>):
List of tuples of doc_types and matching descriptors
always_recreate:
Delete index if already exists (otherwise just update mapping)
reindex:
On mapping mismath, automatically create
new index and migrate existing indexes to it
mapping_generator_cls:
subclass of MappingGenerator
index_settings:
settings which will be used in index creation
"""
existing_index_names = []
if self.__es.indices.exists_alias(name=bucket):
Expand Down Expand Up @@ -142,10 +153,8 @@ def create(self, bucket, doc_types,
def delete(self, bucket=None):
"""Delete index with mapping by schema.
Parameters
----------
bucket: str
Name of index to delete
# Arguments
bucket(str): Name of index to delete
"""
def internal_delete(bucket):
Expand Down

0 comments on commit 7c06a18

Please sign in to comment.