Skip to content

Commit

Permalink
Added examples to all tools
Browse files Browse the repository at this point in the history 
Squashed commit of the following:

commit 5d53911
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Wed Jul 29 15:53:01 2015 -0400

    finished adding help text to all tools

commit ed44c52
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Wed Jul 29 15:40:35 2015 -0400

    added docs to sigma_edit

commit 7e77e09
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Wed Jul 29 15:15:58 2015 -0400

    removed scatter matrix from setup.py

commit 60ad5e9
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Wed Jul 29 15:15:07 2015 -0400

    removed scatter matrix tool

commit 9ff0889
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Wed Jul 29 15:13:35 2015 -0400

    add docs to p.regress

commit f2bf17e
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Wed Jul 29 14:42:12 2015 -0400

    added examples to p.regplot

commit 6cff5bc
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Wed Jul 29 13:12:36 2015 -0400

    fixed failing tests

commit 5b93290
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Tue Jul 28 22:55:13 2015 -0400

    changed how backend is being set

commit d533bcc
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Tue Jul 28 22:28:35 2015 -0400

    added nice error when no data read

commit c1289d5
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 23:52:45 2015 -0400

    readme

commit ab76959
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 23:47:25 2015 -0400

    readme

commit 78d4eb2
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 23:44:29 2015 -0400

    readme

commit 1343d8f
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 23:42:46 2015 -0400

    readme

commit 3d7e0ca
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 23:39:42 2015 -0400

    readme

commit f2a7559
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 23:35:54 2015 -0400

    readme

commit 9c09f80
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 23:35:19 2015 -0400

    readme

commit b8420ed
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 23:30:43 2015 -0400

    readme

commit f94e4e2
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 23:14:44 2015 -0400

    readme

commit 5ad0d62
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 22:53:08 2015 -0400

    readme

commit 84eac30
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 22:50:56 2015 -0400

    readme

commit f78638d
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 22:49:38 2015 -0400

    readme

commit 714b238
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 22:49:08 2015 -0400

    readme

commit 0933b94
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 22:47:50 2015 -0400

    readme

commit 743f277
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 22:47:00 2015 -0400

    readme

commit 7b04631
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 22:45:45 2015 -0400

    readme

commit 0c58992
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 22:43:59 2015 -0400

    readme

commit cc43d4c
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 22:42:14 2015 -0400

    readme

commit 6cdb052
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 20:01:45 2015 -0400

    readme

commit 53b0d84
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 20:00:31 2015 -0400

    readme

commit 05a0b74
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 19:59:47 2015 -0400

    readme

commit 30ef4aa
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 19:58:25 2015 -0400

    readme

commit a605134
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 19:57:47 2015 -0400

    readme

commit d1b6031
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 19:57:08 2015 -0400

    readme

commit da83b5c
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 19:56:26 2015 -0400

    readme

commit ec29fe4
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 19:55:23 2015 -0400

    readme

commit 804412d
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 19:50:18 2015 -0400

    readme

commit 38e499c
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 19:48:38 2015 -0400

    readme

commit 403afb2
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 19:43:03 2015 -0400

    readme

commit fd86313
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 19:34:02 2015 -0400

    readme

commit 78eb865
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 17:32:31 2015 -0400

    readme

commit 034b5f7
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 17:30:31 2015 -0400

    readme

commit afabd5a
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 17:30:01 2015 -0400

    readme

commit 8f75c6f
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 17:27:43 2015 -0400

    readme

commit 5be209b
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 17:24:34 2015 -0400

    readme

commit b41267b
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 17:23:33 2015 -0400

    readme

commit 168a4e4
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 17:22:44 2015 -0400

    readme

commit a058a12
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 17:20:41 2015 -0400

    readme

commit b8df453
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 17:20:00 2015 -0400

    readme

commit 4c6419e
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 17:14:28 2015 -0400

    readme

commit 07ee50e
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 17:12:39 2015 -0400

    readme

commit 1ed5774
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 17:06:30 2015 -0400

    readme

commit 7b90a3d
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:56:33 2015 -0400

    readme

commit 7a741bf
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:55:33 2015 -0400

    readme

commit 49c7cf2
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:50:26 2015 -0400

    readme

commit c530726
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:49:17 2015 -0400

    readme

commit e05fb70
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:37:19 2015 -0400

    readme

commit 2ac969d
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:31:10 2015 -0400

    readme

commit 6cfb58a
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:29:46 2015 -0400

    readme

commit e4ff39b
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:27:16 2015 -0400

    readme

commit 4e38768
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:20:22 2015 -0400

    readme

commit 06216bd
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:19:00 2015 -0400

    readme

commit 5fd6eb0
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:17:49 2015 -0400

    readme

commit 634d484
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:16:35 2015 -0400

    readme

commit d595252
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:14:20 2015 -0400

    readme

commit ced624a
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:08:56 2015 -0400

    readme

commit 47b74fc
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:07:19 2015 -0400

    backup

commit 7683fd8
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:06:42 2015 -0400

    tweaking

commit a84a0b7
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 16:04:45 2015 -0400

    entered tool descriptions

commit 2e3de4d
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 14:37:59 2015 -0400

    backup

commit 1cfbd3e
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 14:36:48 2015 -0400

    added p.hist plot

commit 3faf356
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 13:57:41 2015 -0400

    added facet plot

commit eeee710
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 13:50:11 2015 -0400

    readme writing

commit 2993aca
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 13:45:43 2015 -0400

    backup

commit 28f767c
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 12:30:48 2015 -0400

    backup

commit db8fb06
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 12:28:26 2015 -0400

    backup

commit d25d033
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 12:18:44 2015 -0400

    starting to work on readme

commit ab3efb9
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Mon Jul 27 12:00:34 2015 -0400

    starting to update readme

commit 649ded9
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sun Jul 26 23:05:04 2015 -0400

    allowed for individual trace styles and alphas in p.plo

commit 716d126
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sun Jul 26 22:22:25 2015 -0400

    added examples to p.parallel

commit cbfd679
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sun Jul 26 14:23:05 2015 -0400

    start with examples on p.parallel

commit 1f45db8
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sun Jul 26 14:20:51 2015 -0400

    added examples to p.merge

commit bd1e3b1
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sun Jul 26 14:05:38 2015 -0400

    Merged in ability for plots to have log scale
    Squashed commit of the following:

    commit a017d5a
    Author: Rob deCarvalho <robdmc@gmail.com>
    Date:   Sun Jul 26 14:05:09 2015 -0400

        got log tests passing at 100

commit 943f8ee
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sun Jul 26 13:44:24 2015 -0400

    start writing examples at p.merge

commit dbcd0e9
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sun Jul 26 13:42:53 2015 -0400

    added examples to p.lomb_scargle

commit 4eba0ac
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sun Jul 26 13:01:53 2015 -0400

    added examples to linspace

commit 99c99b5
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sun Jul 26 12:53:29 2015 -0400

    added examples to p.hist

commit e84db7f
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sun Jul 26 12:33:45 2015 -0400

    added examples for p_format

commit 9de984b
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sun Jul 26 12:19:24 2015 -0400

    added examples for p_example_data.py

commit 645a4a0
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sun Jul 26 10:42:10 2015 -0400

    backup

commit b4f8006
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sat Jul 25 23:29:44 2015 -0400

    wrote pandas docs

commit d463296
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sat Jul 25 22:47:33 2015 -0400

    backup

commit 4726c9e
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sat Jul 25 17:41:10 2015 -0400

    added docs makefile

commit 995be37
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sat Jul 25 13:39:21 2015 -0400

    changed p_regress naming to work better with plots

commit 6df27f1
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sat Jul 25 12:35:33 2015 -0400

    backup

commit 757cd49
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Sat Jul 25 11:24:22 2015 -0400

    got lombscargle lib passing at 100% coverage

commit f4cd0ee
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Fri Jul 24 22:47:41 2015 -0400

    added beginnings of lomb-scargle tool

commit f176aff
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Fri Jul 24 22:25:18 2015 -0400

    backup

commit d31ab00
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Fri Jul 24 22:23:21 2015 -0400

    adding lomb scargle code

commit 0033d62
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Fri Jul 24 18:16:43 2015 -0400

    Merged revamping of example data and created p.facet_grid
    Squashed commit of the following:

    commit 9bf27b0
    Author: Rob deCarvalho <robdmc@gmail.com>
    Date:   Fri Jul 24 18:14:22 2015 -0400

        Merged changes that created p.facet_grid
        Squashed commit of the following:

        commit a2ce37e
        Author: Rob deCarvalho <robdmc@gmail.com>
        Date:   Fri Jul 24 18:13:29 2015 -0400

            got p.facet_grid docstring written.

        commit b99c6cb
        Author: Rob deCarvalho <robdmc@gmail.com>
        Date:   Fri Jul 24 17:15:58 2015 -0400

            flake8 fixes

        commit e53fc9a
        Author: Rob deCarvalho <robdmc@gmail.com>
        Date:   Fri Jul 24 17:14:41 2015 -0400

            added facet_grid tests

        commit 9bf4a45
        Author: Rob deCarvalho <robdmc@gmail.com>
        Date:   Fri Jul 24 14:34:16 2015 -0400

            in the process of writing p_facet_grid tests

        commit 5d58c76
        Author: Rob deCarvalho <robdmc@gmail.com>
        Date:   Fri Jul 24 14:22:07 2015 -0400

            need to write tests for p_facet_grid before unwinding this yak

        commit 307fe90
        Author: Rob deCarvalho <robdmc@gmail.com>
        Date:   Fri Jul 24 14:06:53 2015 -0400

            flake8 fixes

        commit 954e90c
        Author: Rob deCarvalho <robdmc@gmail.com>
        Date:   Fri Jul 24 13:43:23 2015 -0400

            got tests passing at 100% again

        commit 4fbc8c1
        Author: Rob deCarvalho <robdmc@gmail.com>
        Date:   Fri Jul 24 13:30:27 2015 -0400

            got arglib tests passing again

        commit 86c625d
        Author: Rob deCarvalho <robdmc@gmail.com>
        Date:   Fri Jul 24 13:00:45 2015 -0400

            reduced the number of backend options

        commit 32443b7
        Author: Rob deCarvalho <robdmc@gmail.com>
        Date:   Thu Jul 23 22:00:53 2015 -0400

            add tests to facet grid. finish sample data, then back to docs

        commit 3ed0401
        Author: Rob deCarvalho <robdmc@gmail.com>
        Date:   Thu Jul 23 21:57:30 2015 -0400

            added backend as configuration option

        commit 5849109
        Author: Rob deCarvalho <robdmc@gmail.com>
        Date:   Thu Jul 23 16:05:29 2015 -0400

            still writing facet grid.  Then need to think about sample data and
            finally docs udpates

        commit 26ed17d
        Author: Rob deCarvalho <robdmc@gmail.com>
        Date:   Thu Jul 23 15:58:36 2015 -0400

            added beginnings of facet_grid code

        commit 8696396
        Author: Rob deCarvalho <robdmc@gmail.com>
        Date:   Thu Jul 23 14:10:18 2015 -0400

            added initial version of facet_grid

    commit 92c2fd3
    Author: Rob deCarvalho <robdmc@gmail.com>
    Date:   Thu Jul 23 14:06:56 2015 -0400

        added electoral_college data

    commit eae373e
    Author: Rob deCarvalho <robdmc@gmail.com>
    Date:   Thu Jul 23 13:53:32 2015 -0400

        changed up available example data files

    commit bcb907b
    Author: Rob deCarvalho <robdmc@gmail.com>
    Date:   Wed Jul 22 19:38:37 2015 -0400

        need to finish p.example_data then go back to p.df docs

commit 1395360
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Wed Jul 22 19:28:29 2015 -0400

    in the process of writing p.df docs, but pausing because I need sample data

commit 5dc315c
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Wed Jul 22 17:15:23 2015 -0400

    created p.crypt help strings

commit a2fff33
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Wed Jul 22 15:37:56 2015 -0400

    finished p.config helptext

commit 9aa4537
Author: Rob deCarvalho <robdmc@gmail.com>
Date:   Wed Jul 22 15:08:37 2015 -0400

    refactored decorating and io options into groups.  got p.cdf looking right.
  • Loading branch information
@robdmc
robdmc committed Jul 29, 2015
1 parent ca5aeb9 commit dd62ffc
Show file tree
Hide file tree
Showing 52 changed files with 3,512 additions and 39,979 deletions.
390 changes: 234 additions & 156 deletions README.md
View file

Large diffs are not rendered by default.

Binary file added images/cdf_plot.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/facet_plot.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed images/gender_bar.png
View file
Binary file not shown.
Binary file added images/hist.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/regplot.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/spectrum.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/timeseries.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed images/tip_vs_bill.png
View file
Binary file not shown.
Binary file added images/tips_vs_bill.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
25 changes: 20 additions & 5 deletions pandashells/bin/p_cdf.py
View file
Expand Up @@ -2,13 +2,14 @@

# standard library imports
import argparse
import textwrap

import sys # NOQA need this for mock testig

from pandashells.lib import module_checker_lib, arg_lib, io_lib, plot_lib

# import required dependencies
modulesOkay = module_checker_lib.check_for_modules([
module_checker_lib.check_for_modules([
'pandas',
'numpy',
'matplotlib',
Expand All @@ -22,12 +23,25 @@


def main():
msg = "Plot cumulative distribution of input column."
msg = textwrap.dedent(
"""
Plots the emperical cumulative distribution function (ECDF).
# read command line arguments
parser = argparse.ArgumentParser(description=msg)
-----------------------------------------------------------------------
Examples:
* Plot ECDF for 10k samples from the standard normal distribution.
p.rand -t normal -n 10000 | p.cdf -c c0
arg_lib.add_args(parser, 'io_in', 'io_out', 'example', 'decorating')
* Instead of plotting, send ECDF values to stdout
p.rand -t normal -n 10000 | p.cdf -c c0 -q | head
-----------------------------------------------------------------------
"""
)

# read command line arguments
parser = argparse.ArgumentParser(
formatter_class=argparse.RawDescriptionHelpFormatter, description=msg)

# specify column to use
parser.add_argument(
Expand All @@ -41,6 +55,7 @@ def main():
help='Quiet mean no plots. Send numeric output to stdout instead')

# parse arguments
arg_lib.add_args(parser, 'decorating', 'io_in', 'io_out',)
args = parser.parse_args()

# get the input dataframe and extract column
Expand Down
28 changes: 23 additions & 5 deletions pandashells/bin/p_config.py
View file
Expand Up @@ -4,6 +4,7 @@
import os
import sys # noqa Not used in code, but patched in testing
import argparse
import textwrap

from pandashells.lib import config_lib

Expand All @@ -12,14 +13,31 @@ def main():
# --- read in the current configuration
default_dict = config_lib.get_config()

#TODO: maybe make this more informative.
msg = (
'Updates file ~/.pandashells, which hold pandashells configuration '
'information.'
msg = textwrap.dedent(
"""
Sets the default IO and plotting behavior for the pandashells toolset
by creating/modifying a hidden configuration file (~/.pandashells)
-----------------------------------------------------------------------
Examples:
* Show the current configuration
p.config
* Set the configuration to "factory defaults"
p.config --force_defaults
* Set default input/output to assume white-space delimited columns
with no headers.
p.config --io_input_header noheader --io_input_type table
p.config --io_output_header noheader --io_output_type table
-----------------------------------------------------------------------
"""
)

# --- populate the arg parser with current configuration
parser = argparse.ArgumentParser(description=msg)
parser = argparse.ArgumentParser(
formatter_class=argparse.RawDescriptionHelpFormatter, description=msg)
parser.add_argument('--force_defaults', action='store_true',
dest='force_defaults',
help='Force to default settings')
Expand Down
28 changes: 22 additions & 6 deletions pandashells/bin/p_crypt.py
View file
@@ -1,19 +1,35 @@
#! /usr/bin/env python

import argparse
import textwrap
import os
import sys

from pandashells.lib import arg_lib


def main():
msg = "Encrypt a file with aes-256-cbc as implemented by openssl. "
msg = textwrap.dedent(
"""
Encrypts and decrypts files using openssl. Openssl provides command-
line tools that are capable of encrypting and decrypting files. This
tool provides a thin wrapper around that capability by hardcoding
the encyrption to be aes256-cbc.
-----------------------------------------------------------------------
Examples:
* Encrypt an input file echoing the openssl command
echo 'plain text' > file.txt
p.crypt -i file.txt -v -o file.txt.crypt
* Decrypt an input file
p.crypt -d -i file.txt.crypt -o file_restored.txt
-----------------------------------------------------------------------
"""
)

# read command line arguments
parser = argparse.ArgumentParser(description=msg)

arg_lib.add_args(parser, 'example')
parser = argparse.ArgumentParser(
formatter_class=argparse.RawDescriptionHelpFormatter, description=msg)

parser.add_argument(
'-i', '--in_file', nargs=1, type=str,
Expand Down
109 changes: 93 additions & 16 deletions pandashells/bin/p_df.py
View file
Expand Up @@ -2,9 +2,11 @@

# standard library imports
import argparse
import textwrap
import os # noqa
import re # noqa
import sys # noqa
import datetime # noqa

from pandashells.lib import module_checker_lib, arg_lib, io_lib

Expand Down Expand Up @@ -58,15 +60,16 @@ def get_modules_and_shortcuts(command_list):
for (module, shortcut) in get_modules_and_shortcuts(sys.argv):
exec('import {} as {}'.format(module, shortcut))

#TODO: change how tests are done to remove this funiness

# TODO: change how tests are done to remove this funiness
# This branch is run in the integrations tests, but since it's being
# run from a system call, coverage doesn't know about it. I'm
# labeling it as no_cover because it actually does get run.
if needs_plots(sys.argv): # pragma: no cover
from pandashells.lib import plot_lib


#TODO: same as above
# TODO: same as above
# This function is run in the integrations tests, but since it's being
# run from a system call, coverage doesn't know about it. I'm
# labeling it as no_cover because it actually does get run.
Expand Down Expand Up @@ -119,28 +122,102 @@ def process_command(args, cmd, df):
return df


#TODO: same as above
# TODO: same as above
# This function is run in the integrations tests, but since it's being
# run from a system call, coverage doesn't know about it. I'm
# labeling it as no_cover because it actually does get run.
def main(): # pragma: no cover
# read command line arguments
msg = (
"Bring pandas manipulation to command line. Input from stdin "
"is placed into a dataframe named 'df'. The output of each "
"command must evaluate to either a dataframe or a series."
"The output of each command will be available to the next command "
"as 'df'. The output of the final command will be sent "
"to stdout. The namespace in which the commands are executed "
"includes pandas as pd, numpy as np, scipy as scp, pylab as pl, "
"dateutil.parser.parse as parse, datetime. Plot-specific "
"commands will be ignored unless a supplied command creates "
"a plot. "
msg = textwrap.dedent(
"""
Enables pandas dataframe processing at the unix command line.
This is the real workhorse of the pandashells toolkit. It reads data
from stdin as a dataframe, which is passed through any number of pandas
operations provided on the command line. Output is always to stdout.
Each operation assumes data is in a dataframe named df. Operations
performed on this dataframe will overwrite the df variable with
the results of that operation. Special consideration is taken for
assignments such as df['a'] = df.b + df.c. These are understood
to agument the input dataframe with a new column. By way of example,
this command:
p.df 'df.groupby(by="a").b.count()' 'df.reset_index()'
is equivalent to the python expressions:
df = df.groupby(by="a").b.count()
df = df.reset_index()
In addition to providing access to pandas dataframes, a number of
modules are loaded into the namespace so as to be accessible from the
command line. These modules are:
pd = pandas
np = numpy
scp = scipy
pl = pylab
parse = dateutil.parser.parse
datetime = datetime
re = re
** Important **
When creating chains of dataframe operations (see examples), it is
important to express your chain of operations before any options. This
is because some options can take multiple arguments and the parser
won't be able to properly decode your meaning.
For example:
cat file.csv | p.df 'df["x"] = df.y + 1' -o table noheader # GOOD
cat file.csv | p.df -o table noheader 'df["x"] = df.y + 1' # BAD
Input can be read in different formats as specified by the -i switch.
The most common formats are csv and table (white-space-delimited). In
either of these formats, p.df can accomodate input data that either
does or doesn not have a header row. When no header row is indicated,
The columns of the Dataframe will be labeled as c0, c1, ..., cN.
Plotting methods invoked on a Dataframe generate no output, but
create an interactive plot instead. There are a number of plot
specific options available at the command line that govern the details
of how these plots are rendered (e.g. --xlim, --legend, etc).
-----------------------------------------------------------------------
Examples:
* Print a csv file in nice tabular format
p.example_data -d tips | p.df -o table | head
* Select by row
p.example_data -d tips \\
| p.df 'df[df.sex=="Female"]' 'df[df.smoker=="Yes"]' -o table
* Extract columns
p.example_data -d tips \\
| p.df 'df[["total_bill", "tip"]].head()' -o table
* Perform grouped aggregations
p.example_data -d tips | p.df \\
'df.groupby(by=["sex", "smoker"]).tip.sum()' -o table index
* Use pandas plotting methods
p.example_data -d tips | p.df \\
'df.groupby(by="day").total_bill.sum().plot(kind="barh")'\\
--xlabel 'Dollars' --title 'Total Bills by Day'
* Convert between tabular and csv format with/without header rows
seq 10 | awk '{print $1, 2*$1}'\\
| p.df --names a b -i table noheader | p.df -o table noheader
-----------------------------------------------------------------------
"""
)
parser = argparse.ArgumentParser(description=msg)

parser = argparse.ArgumentParser(
formatter_class=argparse.RawDescriptionHelpFormatter, description=msg)
arg_lib.add_args(parser, 'io_in', 'io_out', 'decorating', 'example')
msg = (
'(MUST come before any options) '
'[statement ...] Statement(s) to execute. '
)
parser.add_argument(
"statement", help="[statement ...] Statement(s) to execute", nargs="*")
"statement", help=msg, nargs="*")
args = parser.parse_args()

# get the input dataframe
Expand Down
69 changes: 69 additions & 0 deletions pandashells/bin/p_example_data.py
View file
@@ -0,0 +1,69 @@
#! /usr/bin/env python

# standard library imports
import os
import sys # noqa
import argparse
import textwrap

import pandashells


def main():
# create a dict of data-set names and corresponding files
package_dir = os.path.dirname(os.path.realpath(pandashells.__file__))
sample_data_dir = os.path.realpath(os.path.join(package_dir, 'example_data'))

f_dict = {}
for f in os.listdir(sample_data_dir):
f_dict[f.replace('.csv', '')] = os.path.join(sample_data_dir, f)

# read command line arguments
msg = textwrap.dedent(
"""
Provides access to sample csv data sets for exploring the pandashells
toolkit.
-----------------------------------------------------------------------
Examples:
* Restaraunt tips along with patron information.
p.example_data -d tips | head
* Relative rise in global sea surface height over the past couple
decades. Original source: http://sealevel.colorado.edu/
p.example_data -d sealevel | head
* Polling data for 2008 US presidential
p.example_data -d election | head
* US Electoral college and population numbers by state
p.example_data -d electoral_college | head
-----------------------------------------------------------------------
"""
)

parser = argparse.ArgumentParser(
formatter_class=argparse.RawDescriptionHelpFormatter, description=msg)

parser.add_argument(
'-d', '--dataset', nargs=1, type=str,
dest='dataset', choices=sorted(f_dict.keys()), required=True,
help='The name of the sample dataset')

# parse arguments
args = parser.parse_args()

# print contents of data file to output
f_name = f_dict[args.dataset[0]]
with open(f_name) as in_file:
try:
# line by line avoids weird sys.excepthook bug on pipe to head
for line in in_file:
print line.strip()
except IOError:
pass


if __name__ == '__main__': # pragma: no cover
main()

0 comments on commit dd62ffc

Please sign in to comment.