Skip to content

Commit

Permalink
update iterator
Browse files Browse the repository at this point in the history
  • Loading branch information
phrocker committed May 16, 2020
1 parent 4222135 commit 596dfb3
Show file tree
Hide file tree
Showing 3 changed files with 36 additions and 2 deletions.
21 changes: 21 additions & 0 deletions docs/hdfsclient.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
.. image:: https://camo.githubusercontent.com/dbf39cef1a973d8741437693e96b59e31d9e3754/68747470733a2f2f7777772e736861726b626974652e696f2f77702d636f6e74656e742f75706c6f6164732f323031372f30322f736861726b626974652e6a7067

HDFS Client
==================

The hdfs client only supports `mkdir <https://docs.sharkbite.io/en/latest/sharkbitedocs.html#pysharkbite.Hdfs.mkdir>`_ and
`list <https://docs.sharkbite.io/en/latest/sharkbitedocs.html#pysharkbite.Hdfs.list>`_ at this time. Once additional tests are built
around the remainder of the HDFS python functionality we will add in the remaining portions of the client.

An example usage of these functions is below. Note that if Opening RFiles in pysharkbite, you must specify the full path including
the hdfs protocol if it is located on HDFS. This will open a full HDFS client to access these files.

.. code-block:: python
import pysharkbite
hdfs = pysharkbite.Hdfs("namenode",8020);
hdfs.mkdir("/directoryA/directoryB");
hdfs.list("/");
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Welcome to Sharkbite's documentation!
:caption: Examples

asynciterator
hdfsclient

.. toctree::
:maxdepth: 2
Expand Down
16 changes: 14 additions & 2 deletions docs/intro.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,16 +33,28 @@ This python client can be installed via `pip install sharkbite`
`A Python example <https://github.com/phrocker/sharkbite/blob/master/examples/pythonexample.py>`_ is included. This is your primary example of the Python bound sharkbite
library.

Sharkbite now supports async iteration `A simple example <https://github.com/phrocker/sharkbite/blob/master/examples/asyncexample.py>`_ is provided.
**Sharkbite** supports async iteration `A simple example <https://github.com/phrocker/sharkbite/blob/master/examples/asyncexample.py>`_ is provided.

Features
********

HDFS Client
************

**Sharkbite** supports a limited HDFS client. As this functionality grows so will the capabilities. Version 0.7 will support a complete
HDFS client. Since Sharkbite it built as a python bindings around a C++ Client, the python client will mature slightly behind the C++ client,
hence the delta with building this into V 0.7

Version Detection
************

**Sharkbite** detects the version of Apache Accumulo. Therefore you will be able to simply create a connector to the zookeeper instance.


Hedged Reads
************

Sharkbite supports hedged reads ( executing scans against RFiles when they can be accessed ) concurrently with
**Sharkbite** supports hedged reads ( executing scans against RFiles when they can be accessed ) concurrently with
Accumulo RPC scans. The first executor to complete will return your results. This feature is in beta and not suggested
for production environments.

Expand Down

0 comments on commit 596dfb3

Please sign in to comment.