Skip to content

Commit

Permalink
Update all utility usage messages in docs, closes #430
Browse files Browse the repository at this point in the history
  • Loading branch information
James McKinney committed Jan 23, 2016
1 parent 1866f8f commit d3c17f0
Show file tree
Hide file tree
Showing 16 changed files with 77 additions and 58 deletions.
5 changes: 3 additions & 2 deletions docs/common_arguments.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,13 @@ All utilities which accept CSV as input share a set of common command-line argum
-z MAXFIELDSIZE, --maxfieldsize MAXFIELDSIZE
Maximum length of a single field in the input CSV
file.
-H, --no-header-row Specifies that the input CSV file has no header row.
-e ENCODING, --encoding ENCODING
Specify the encoding the input CSV file.
-S, --skipinitialspace
Ignore whitespace immediately following the delimiter.
-H, --no-header-row Specifies that the input CSV file has no header row.
Will create default headers.
-v, --verbose Print detailed tracebacks when errors occur.
Specify the encoding the input file.
-l, --linenumbers Insert a column of line numbers at the front of the
output. Useful when piping to grep or as a simple
primary key.
Expand Down
10 changes: 5 additions & 5 deletions docs/scripts/csvclean.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,20 @@ Description
Cleans a CSV file of common syntax errors. Outputs [basename]_out.csv and [basename]_err.csv, the former containing all valid rows and the latter containing all error rows along with line numbers and descriptions::

usage: csvclean [-h] [-d DELIMITER] [-t] [-q QUOTECHAR] [-u {0,1,2,3}] [-b]
[-p` ESCAPECHAR] [-e ENCODING] [-n]
[-p ESCAPECHAR] [-z MAXFIELDSIZE] [-e ENCODING] [-S] [-v] [-l]
[--zero] [-n]
[FILE]

Fix common syntax errors in a CSV file.
Fix common errors in a CSV file.

positional arguments:
FILE The CSV file to operate on. If omitted, will accept
input on STDIN.

optional arguments:
-h, --help show this help message and exit
-n, --dry-run If this argument is present, no output will be
created. Information about what would have been done
will be printed to STDERR.
-n, --dry-run Do not create output files. Information about what
would have been done will be printed to STDERR.

See also: :doc:`../common_arguments`.

Expand Down
3 changes: 2 additions & 1 deletion docs/scripts/csvcut.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ Description
Filters and truncates CSV files. Like unix "cut" command, but for tabular data::

usage: csvcut [-h] [-d DELIMITER] [-t] [-q QUOTECHAR] [-u {0,1,2,3}] [-b]
[-p` ESCAPECHAR] [-e ENCODING] [-n] [-c COLUMNS] [-s] [-l]
[-p ESCAPECHAR] [-z MAXFIELDSIZE] [-e ENCODING] [-S] [-H] [-v]
[-l] [--zero] [-n] [-c COLUMNS] [-C NOT_COLUMNS] [-x]
[FILE]

Filter and truncate CSV files. Like unix "cut" command, but for tabular data.
Expand Down
3 changes: 2 additions & 1 deletion docs/scripts/csvformat.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ Convert a CSV file to a custom output format.::
input on STDIN.

optional arguments:
-h, --help show this help message and exit
-D OUT_DELIMITER, --out-delimiter OUT_DELIMITER
Delimiting character of the output CSV file.
-T, --out-tabs Specifies that the output CSV file is delimited with
Expand All @@ -41,7 +42,7 @@ Convert a CSV file to a custom output format.::
specified.
-M OUT_LINETERMINATOR, --out-lineterminator OUT_LINETERMINATOR
Character used to terminate lines in the output CSV
file.
file.

See also: :doc:`../common_arguments`.

Expand Down
8 changes: 5 additions & 3 deletions docs/scripts/csvgrep.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,12 @@ Description
Filter tabular data to only those rows where certain columns contain a given value or match a regular expression::

usage: csvgrep [-h] [-d DELIMITER] [-t] [-q QUOTECHAR] [-u {0,1,2,3}] [-b]
[-p ESCAPECHAR] [-e ENCODING] [-l] [-n] [-c COLUMNS] [-r]
[FILE] [PATTERN]
[-p ESCAPECHAR] [-z MAXFIELDSIZE] [-e ENCODING] [-S] [-v] [-l]
[--zero] [-n] [-c COLUMNS] [-m PATTERN] [-r REGEX]
[-f MATCHFILE] [-i]
[FILE]

Like the unix "grep" command, but for tabular data.
Search CSV files. Like the unix "grep" command, but for tabular data.

positional arguments:
FILE The CSV file to operate on. If omitted, will accept
Expand Down
8 changes: 4 additions & 4 deletions docs/scripts/csvjoin.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ Description
Merges two or more CSV tables together using a method analogous to SQL JOIN operation. By default it performs an inner join, but full outer, left outer, and right outer are also available via flags. Key columns are specified with the -c flag (either a single column which exists in all tables, or a comma-seperated list of columns with one corresponding to each). If the columns flag is not provided then the tables will be merged "sequentially", that is they will be merged in row order with no filtering::

usage: csvjoin [-h] [-d DELIMITER] [-t] [-q QUOTECHAR] [-u {0,1,2,3}] [-b]
[-p` ESCAPECHAR] [-e ENCODING] [--outer] [--left]
[--right]
FILES [FILES ...]
[-p ESCAPECHAR] [-z MAXFIELDSIZE] [-e ENCODING] [-S] [-v] [-l]
[--zero] [-c COLUMNS] [--outer] [--left] [--right]
[FILE [FILE ...]]

Execute a SQL-like join to merge CSV files on a specified column or columns.

positional arguments:
FILES The CSV files to operate on. If only one is specified,
FILE The CSV files to operate on. If only one is specified,
it will be copied to STDOUT.

optional arguments:
Expand Down
5 changes: 3 additions & 2 deletions docs/scripts/csvjson.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ Description
Converts a CSV file into JSON or GeoJSON (depending on flags)::

usage: csvjson [-h] [-d DELIMITER] [-t] [-q QUOTECHAR] [-u {0,1,2,3}] [-b]
[-p ESCAPECHAR] [-z MAXFIELDSIZE] [-e ENCODING] [-H] [-v] [-l]
[-p ESCAPECHAR] [-z MAXFIELDSIZE] [-e ENCODING] [-S] [-v] [-l]
[--zero] [-i INDENT] [-k KEY] [--lat LAT] [--lon LON]
[--crs CRS]
[--crs CRS] [--stream]
[FILE]

Convert a CSV file into JSON (or GeoJSON).
Expand All @@ -20,6 +20,7 @@ Converts a CSV file into JSON or GeoJSON (depending on flags)::
input on STDIN.

optional arguments:
-h, --help show this help message and exit
-i INDENT, --indent INDENT
Indent the output JSON this many spaces. Disabled by
default.
Expand Down
3 changes: 2 additions & 1 deletion docs/scripts/csvlook.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ Description
Renders a CSV to the command line in a readable, fixed-width format::

usage: csvlook [-h] [-d DELIMITER] [-t] [-q QUOTECHAR] [-u {0,1,2,3}] [-b]
[-p` ESCAPECHAR] [-e ENCODING]
[-p ESCAPECHAR] [-z MAXFIELDSIZE] [-e ENCODING] [-S] [-H] [-v]
[-l] [--zero]
[FILE]

Render a CSV file in the console as a fixed-width table.
Expand Down
18 changes: 10 additions & 8 deletions docs/scripts/csvpy.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,20 +5,22 @@ csvpy
Description
===========

Loads a CSV file into a :class:`csvkit.CSVKitReader` object and then drops into a Python shell so the user can inspect the data however they see fit::
Loads a CSV file into a :class:`agate.Reader` object and then drops into a Python shell so the user can inspect the data however they see fit::

usage: csvpy [-h] [-d DELIMITER] [-t] [-q QUOTECHAR] [-u {0,1,2,3}] [-b]
[-p ESCAPECHAR] [-z MAXFIELDSIZE] [-e ENCODING] [-H] [-v]
FILE
[-p ESCAPECHAR] [-z MAXFIELDSIZE] [-e ENCODING] [-S] [-v]
[--dict]
[FILE]

Load a CSV file into a CSVKitReader object and then drops into a Python shell.
Load a CSV file into a CSV reader and then drop into a Python shell.

positional arguments:
FILE The CSV file to operate on.
FILE The CSV file to operate on. If omitted, will accept
input on STDIN.

optional arguments:
-h, --help show this help message and exit
--dict Use CSVKitDictReader instead of CSVKitReader.
--dict Use a CSV DictReader instead of a normal reader.

This utility will automatically use the IPython shell if it is installed, otherwise it will use the running Python shell.

Expand All @@ -34,14 +36,14 @@ Examples
Basic use::

$ csvpy examples/dummy.csv
Welcome! "examples/dummy.csv" has been loaded in a CSVKitReader object named "reader".
Welcome! "examples/dummy.csv" has been loaded in a reader object named "reader".
>>> reader.next()
[u'a', u'b', u'c']

As a dictionary::

$ csvpy --dict examples/dummy.csv -v
Welcome! "examples/dummy.csv" has been loaded in a CSVKitDictReader object named "reader".
Welcome! "examples/dummy.csv" has been loaded in a DictReader object named "reader".
>>> reader.next()
{u'a': u'1', u'c': u'3', u'b': u'2'}

7 changes: 4 additions & 3 deletions docs/scripts/csvsort.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ Description
Sort CSV files. Like unix "sort" command, but for tabular data::

usage: csvsort [-h] [-d DELIMITER] [-t] [-q QUOTECHAR] [-u {0,1,2,3}] [-b]
[-p` ESCAPECHAR] [-e ENCODING] [-n] [-c COLUMNS] [-r]
[-p ESCAPECHAR] [-z MAXFIELDSIZE] [-e ENCODING] [-S] [-H] [-v]
[-l] [--zero] [-y SNIFFLIMIT] [-n] [-c COLUMNS] [-r]
[--no-inference]
[FILE]

Sort CSV files. Like unix "sort" command, but for tabular data.
Expand All @@ -22,15 +24,14 @@ Sort CSV files. Like unix "sort" command, but for tabular data::
-y SNIFFLIMIT, --snifflimit SNIFFLIMIT
Limit CSV dialect sniffing to the specified number of
bytes. Specify "0" to disable sniffing entirely.
Specify the encoding the input file.
-n, --names Display column names and indices from the input CSV
and exit.
-c COLUMNS, --columns COLUMNS
A comma separated list of column indices or names to
sort by. Defaults to all columns.
-r, --reverse Sort in descending order.
--no-inference Disable type inference when parsing the input.

See also: :doc:`../common_arguments`.

Examples
Expand Down
30 changes: 16 additions & 14 deletions docs/scripts/csvsql.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,16 @@ Description
Generate SQL statements for a CSV file or execute those statements directly on a database. In the latter case supports both creating tables and inserting data::

usage: csvsql [-h] [-d DELIMITER] [-t] [-q QUOTECHAR] [-u {0,1,2,3}] [-b]
[-p ESCAPECHAR] [-z MAXFIELDSIZE] [-e ENCODING] [-H] [-v]
[-y SNIFFLIMIT]
[-i {access,sybase,sqlite,informix,firebird,mysql,oracle,maxdb,postgresql,mssql}]
[--db CONNECTION_STRING] [--insert]
[FILE]
[-p ESCAPECHAR] [-z MAXFIELDSIZE] [-e ENCODING] [-S] [-H] [-v]
[--zero] [-y SNIFFLIMIT]
[-i {firebird,maxdb,informix,mssql,oracle,sybase,sqlite,access,mysql,postgresql}]
[--db CONNECTION_STRING] [--query QUERY] [--insert]
[--tables TABLE_NAMES] [--no-constraints] [--no-create]
[--blanks] [--no-inference] [--db-schema DB_SCHEMA]
[FILE [FILE ...]]

Generate SQL statements for a CSV file or create execute those statements
directly on a database.

Generate a SQL CREATE TABLE statement for a CSV file.
Generate SQL statements for one or more CSV files, create execute those
statements directly on a database, and execute one or more SQL queries.

positional arguments:
FILE The CSV file(s) to operate on. If omitted, will accept
Expand All @@ -28,7 +28,7 @@ Generate SQL statements for a CSV file or execute those statements directly on a
-y SNIFFLIMIT, --snifflimit SNIFFLIMIT
Limit CSV dialect sniffing to the specified number of
bytes. Specify "0" to disable sniffing entirely.
-i {access,sybase,sqlite,informix,firebird,mysql,oracle,maxdb,postgresql,mssql}, --dialect {access,sybase,sqlite,informix,firebird,mysql,oracle,maxdb,postgresql,mssql}
-i {firebird,maxdb,informix,mssql,oracle,sybase,sqlite,access,mysql,postgresql}, --dialect {firebird,maxdb,informix,mssql,oracle,sybase,sqlite,access,mysql,postgresql}
Dialect of SQL to generate. Only valid when --db is
not specified.
--db CONNECTION_STRING
Expand All @@ -39,16 +39,18 @@ Generate SQL statements for a CSV file or execute those statements directly on a
--insert In addition to creating the table, also insert the
data into the table. Only valid when --db is
specified.
--table TABLE_NAME Specify a name for the table to be created. If
omitted, the filename (minus extension) will be used.
--tables TABLE_NAMES Specify one or more names for the tables to be
created. If omitted, the filename (minus extension) or
"stdin" will be used.
--no-constraints Generate a schema without length limits or null
checks. Useful when sampling big tables.
--no-create Skip creating a table. Only valid when --insert is
specified.
--blanks Do not coerce empty strings to NULL values.
--no-inference Disable type inference when parsing the input.
--db-schema Optional name of database schema to create table(s)
in.
--db-schema DB_SCHEMA
Optional name of database schema to create table(s)
in.

See also: :doc:`../common_arguments`.

Expand Down
8 changes: 5 additions & 3 deletions docs/scripts/csvstack.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,15 @@ Description
Stack up the rows from multiple CSV files, optionally adding a grouping value to each row::

usage: csvstack [-h] [-d DELIMITER] [-t] [-q QUOTECHAR] [-u {0,1,2,3}] [-b]
[-p` ESCAPECHAR] [-e ENCODING] [-g GROUPS] [-n GROUP_NAME]
FILES [FILES ...]
[-p ESCAPECHAR] [-z MAXFIELDSIZE] [-e ENCODING] [-S] [-H] [-v]
[-l] [--zero] [-g GROUPS] [-n GROUP_NAME] [--filenames]
FILE [FILE ...]

Stack up the rows from multiple CSV files, optionally adding a grouping value.

positional arguments:
FILES
FILE The CSV file(s) to operate on. If omitted, will accept
input on STDIN.

optional arguments:
-h, --help show this help message and exit
Expand Down
9 changes: 6 additions & 3 deletions docs/scripts/csvstat.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,13 @@ Description
Prints descriptive statistics for all columns in a CSV file. Will intelligently determine the type of each column and then print analysis relevant to that type (ranges for dates, mean and median for integers, etc.)::

usage: csvstat [-h] [-d DELIMITER] [-t] [-q QUOTECHAR] [-u {0,1,2,3}] [-b]
[-p` ESCAPECHAR] [-e ENCODING]
[FILE]
[-p ESCAPECHAR] [-z MAXFIELDSIZE] [-e ENCODING] [-S] [-H] [-v]
[--zero] [-y SNIFFLIMIT] [-c COLUMNS] [--max] [--min] [--sum]
[--mean] [--median] [--stdev] [--nulls] [--unique] [--freq]
[--len] [--count]
[FILE]

Print descriptive statistics for all columns in a CSV file.
Print descriptive statistics for each column in a CSV file.

positional arguments:
FILE The CSV file to operate on. If omitted, will accept
Expand Down
8 changes: 5 additions & 3 deletions docs/scripts/in2csv.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@ Converting fixed width requires that you provide a schema file with the "-s" opt
The header line is required though the columns may be in any order::

usage: in2csv [-h] [-d DELIMITER] [-t] [-q QUOTECHAR] [-u {0,1,2,3}] [-b]
[-p` ESCAPECHAR] [-e ENCODING] [-f FORMAT] [-s SCHEMA]
[-p ESCAPECHAR] [-z MAXFIELDSIZE] [-e ENCODING] [-S] [-H] [-v]
[-l] [--zero] [-f FILETYPE] [-s SCHEMA] [-k KEY] [-y SNIFFLIMIT]
[--sheet SHEET] [--no-inference]
[FILE]

Convert common, but less awesome, tabular data formats to CSV.
Expand All @@ -28,10 +30,10 @@ The header line is required though the columns may be in any order::

optional arguments:
-h, --help show this help message and exit
-f FORMAT, --format FORMAT
-f FILETYPE, --format FILETYPE
The format of the input file. If not specified will be
inferred from the file type. Supported formats: csv,
dbf, fixed, geojson, json, ndjson, xls, xlsx.
fixed, geojson, json, ndjson, xls, xlsx.
-s SCHEMA, --schema SCHEMA
Specifies a CSV-formatted schema file for converting
fixed-width files. See documentation for details.
Expand Down
6 changes: 3 additions & 3 deletions docs/scripts/sql2csv.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ Description

Executes arbitrary commands against a SQL database and outputs the results as a CSV::

usage: sql2csv [-h] [-v] [-l] [--db CONNECTION_STRING] [-q QUERY] [-H] [FILE]
usage: sql2csv [-h] [-v] [-l] [--db CONNECTION_STRING] [--query QUERY] [-H]
[FILE]

Execute an SQL query on a database and output the result to a CSV file.

Expand All @@ -24,8 +25,7 @@ Executes arbitrary commands against a SQL database and outputs the results as a
--db CONNECTION_STRING
An sqlalchemy connection string to connect to a
database.
--query QUERY
The SQL query to execute. If specified, it overrides
--query QUERY The SQL query to execute. If specified, it overrides
FILE and STDIN.
-H, --no-header-row Do not output column names.

Expand Down
4 changes: 2 additions & 2 deletions docs/tutorial/4_going_elsewhere.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,12 @@ csvpy: going into code
For the programmers out there, the command line is rarely as functional as just writing a little bit of code. :doc:`/scripts/csvpy` exists just to make a programmer's life easier. Invoking it simply launches a Python interactive terminal, with the data preloaded into a CSV reader::

$ csvpy data.csv
Welcome! "data.csv" has been loaded in a CSVKitReader object named "reader".
Welcome! "data.csv" has been loaded in a reader object named "reader".
>>> print len(list(reader))
1037
>>> quit()

In addition to being a time-saver, because this uses csvkit's internal :class:`CSVKitReader` the reader is Unicode aware.
In addition to being a time-saver, because this uses agate, the reader is Unicode aware.

csvformat: for legacy systems
=============================
Expand Down

0 comments on commit d3c17f0

Please sign in to comment.