Skip to content
This repository has been archived by the owner on Jul 23, 2022. It is now read-only.

Commit

Permalink
rst cleaning
Browse files Browse the repository at this point in the history
  • Loading branch information
sdpython committed Dec 28, 2016
1 parent 792a11e commit 5b93ec0
Show file tree
Hide file tree
Showing 7 changed files with 32 additions and 54 deletions.
33 changes: 14 additions & 19 deletions README.rst
@@ -1,5 +1,4 @@


.. _l-README:

README
Expand All @@ -12,62 +11,59 @@ README
.. image:: https://ci.appveyor.com/api/projects/status/67ljkgh36klak07a?svg=true
:target: https://ci.appveyor.com/project/sdpython/pysqllike
:alt: Build Status Windows

.. image:: https://badge.fury.io/py/pysqllike.svg
:target: http://badge.fury.io/py/pysqllike
:target: http://badge.fury.io/py/pysqllike

.. image:: http://img.shields.io/github/issues/sdpython/pysqllike.png
:alt: GitHub Issues
:target: https://github.com/sdpython/pysqllike/issues

.. image:: https://img.shields.io/badge/license-MIT-blue.svg
:alt: MIT License
:target: http://opensource.org/licenses/MIT

.. image:: https://landscape.io/github/sdpython/pysqllike/master/landscape.svg?style=flat
:target: https://landscape.io/github/sdpython/pysqllike/master
:alt: Code Health

.. image:: https://requires.io/github/sdpython/pysqllike/requirements.svg?branch=master
:target: https://requires.io/github/sdpython/pysqllike/requirements/?branch=master
:alt: Requirements Status
:alt: Requirements Status

.. image:: https://codecov.io/github/sdpython/pysqllike/coverage.svg?branch=master
:target: https://codecov.io/github/sdpython/pysqllike?branch=master



**Links:**

* `GitHub/pysqllike <https://github.com/sdpython/pysqllike>`_
* `documentation <http://www.xavierdupre.fr/app/pysqllike/helpsphinx/index.html>`_
* `Blog <http://www.xavierdupre.fr/app/pysqllike/helpsphinx/blog/main_0000.html#ap-main-0>`_


Description
-----------

Writing a map/reduce job
(using `PIG <https://pig.apache.org/>`_ for example),
usually requires to switch from local files to remote files
(on `Hadoop <http://hadoop.apache.org/>`_).
(on `Hadoop <http://hadoop.apache.org/>`_).
On way to work is extract a small sample of the data which will be processed
by a map/reduce job. The job is then locally developped. And when it works,
it is run on a parallized environment.

The goal of this extension is allow the implementation of
The goal of this extension is allow the implementation of
this job using Python syntax as follows:


::

def myjob(input):
iter = input.select (input.age, input.nom, age2 = input.age2*input.age2)
wher = iter.where( (iter.age > 60).Or(iter.age < 25))
return where
return where

input = IterRow (None, [ {"nom": 10}, {"jean": 40} ] )
output = myjob(input)

When the job is ready, it can be translated into a `PIG <https://pig.apache.org/>`_
job::

Expand All @@ -76,15 +72,14 @@ job::
wher = FILTER iter BY age > 60 or age < 25 ;
STORE wher INTO '...' USING PigStorage();

It should also be translated into
It should also be translated into
`SQL <http://fr.wikipedia.org/wiki/Structured_Query_Language>`_.

Functionalities
---------------

* not yet ready


Design
------

Expand Down
4 changes: 1 addition & 3 deletions _doc/sphinxdoc/source/blog/2015/2015-06-06-about.rst
@@ -1,14 +1,12 @@


.. blogpost::
:title: About this module
:keywords: about
:date: 2015-06-06
:categories: teachings

I first intented to use this module to speed up writing
map reduce jobs by using a kind of functional syntax to define the job.
I allow easy testing on local data and some functions to translate
this job into various map reduce languages SQL, PIG, Python.
But I do not have time to really work on it and complete it.

5 changes: 1 addition & 4 deletions _doc/sphinxdoc/source/glossary.rst
@@ -1,13 +1,10 @@

.. index:: glossary


Glossary
========

.. glossary::

pysqllike
this module


37 changes: 14 additions & 23 deletions _doc/sphinxdoc/source/index.rst
Expand Up @@ -6,39 +6,36 @@
pysqllike documentation
=======================


.. image:: https://travis-ci.org/sdpython/pysqllike.svg?branch=master
:target: https://travis-ci.org/sdpython/pysqllike
:alt: Build status

.. image:: https://ci.appveyor.com/api/projects/status/67ljkgh36klak07a?svg=true
:target: https://ci.appveyor.com/project/sdpython/pysqllike
:alt: Build Status Windows

.. image:: https://badge.fury.io/py/pysqllike.svg
:target: http://badge.fury.io/py/pysqllike

.. image:: http://img.shields.io/github/issues/sdpython/pysqllike.png
:alt: GitHub Issues
:target: https://github.com/sdpython/pysqllike/issues

.. image:: https://img.shields.io/badge/license-MIT-blue.svg
:alt: MIT License
:target: http://opensource.org/licenses/MIT

.. image:: https://landscape.io/github/sdpython/pysqllike/master/landscape.svg?style=flat
:target: https://landscape.io/github/sdpython/pysqllike/master
:alt: Code Health

.. image:: https://requires.io/github/sdpython/pysqllike/requirements.svg?branch=master
:target: https://requires.io/github/sdpython/pysqllike/requirements/?branch=master
:alt: Requirements Status
:alt: Requirements Status

.. image:: https://codecov.io/github/sdpython/pysqllike/coverage.svg?branch=master
:target: https://codecov.io/github/sdpython/pysqllike?branch=master



**Links:** `pypi <https://pypi.python.org/pypi/pysqllike/>`_,
`github <https://github.com/sdpython/pysqllike/>`_,
`documentation <http://www.xavierdupre.fr/app/pysqllike/helpsphinx/index.html>`_,
Expand All @@ -47,32 +44,30 @@ pysqllike documentation
:ref:`blog <ap-main-0>`,
:ref:`l-issues-todolist`


What is it?
-----------

Writing a map/reduce job
(using `PIG <https://pig.apache.org/>`_ for example),
usually requires to switch from local files to remote files
(on `Hadoop <http://hadoop.apache.org/>`_).
(on `Hadoop <http://hadoop.apache.org/>`_).
On way to work is extract a small sample of the data which will be processed
by a map/reduce job. The job is then locally developped. And when it works,
it is run on a parallized environment.

The goal of this extension is allow the implementation of
The goal of this extension is allow the implementation of
this job using Python syntax as follows:


::

def myjob(input):
iter = input.select (input.age, input.nom, age2 = input.age2*input.age2)
wher = iter.where( (iter.age > 60).Or(iter.age < 25))
return wher

input = IterRow (None, [ {"nom": 10}, {"jean": 40} ] )
output = myjob(input)

When the job is ready, it can be translated into a `PIG <https://pig.apache.org/>`_
job::

Expand All @@ -81,7 +76,7 @@ job::
wher = FILTER iter BY age > 60 or age < 25 ;
STORE wher INTO '...' USING PigStorage();

It should also be translated into
It should also be translated into
`SQL <http://fr.wikipedia.org/wiki/Structured_Query_Language>`_.

Installation
Expand All @@ -93,18 +88,16 @@ Functionalities
---------------

* not yet ready


Quick start
-----------

.. toctree::
:maxdepth: 1

all_example
all_notebooks



Indices and tables
------------------

Expand All @@ -121,7 +114,5 @@ Navigation

.. toctree::
:maxdepth: 1

indexmenu


indexmenu
2 changes: 1 addition & 1 deletion _doc/sphinxdoc/source/indexmenu.rst
Expand Up @@ -17,4 +17,4 @@ through documentation
license
filechanges
README
all_indexes
all_indexes
3 changes: 0 additions & 3 deletions _doc/sphinxdoc/source/installation.rst
@@ -1,7 +1,4 @@
Installation
============


See `Installation <http://www.xavierdupre.fr/app/pyquickhelper/helpsphinx/installation.html>`_.


2 changes: 1 addition & 1 deletion _doc/sphinxdoc/source/license.rst
Expand Up @@ -4,4 +4,4 @@ License
=======

.. include:: LICENSE.txt
:literal:
:literal:

0 comments on commit 5b93ec0

Please sign in to comment.