2.7.0 Print the SQL query results to STDOUT, implement --logLevel

For any CLI tool, it's a bit weird not to allow printing the output to the standard output. CSV Cruncher is no exception.

Now, the -out option may take - (minus) as a value, directing the CSV data (SQL query result) to STDOUT:

crunch -in chatgpt-alternatives.csv -out -  -sql 'SELECT name FROM chatgpt-alternatives WHERE monthlyPrice < 10'

For now, the output is mixed with the logging output, since CSV Cruncher is still freshly out of prototype phase (after 13 years :) )

However, that can be neglected by turning off the logging:

crunch --logLevel OFF ...

Setting the log level should have worked since 2.4.0, but somehow, the implementation slipped out of that version.

Both features will be improved in the further versions, as poor UX has been identified as the main hindrance of user adoption.

2.6.0: Custom table names for inputs - `-in ... -as ...`

Before 2.6.0, the tables were named after the input file name.

crunch -in SomeVeryLongName1234567890.csv -out output.csv -sql "SELECT * FROM SomeVeryLongName1234567890"

As of 2.6.0, it is possible to set the table name using -as.

crunch -in SomeVeryLongName1234567890.csv -as data -out output.csv -sql "SELECT * FROM data"

It is also now possible to import a file twice, if that's needed (although that should rather be done by a self-JOIN).

crunch -in data.csv -as data1 -in data.csv -as data2 -out output.csv -sql "SELECT * FROM data1 UNION SELECT * FROM data2"

2.5.0: Allow indexes

As of 2.5.0, the imported CSV may be covered by an index by the underlying HSQLDB. This speeds up joins across large data sets significantly. Example: 10.000 x 10.000 rows join took around 30 minutes. With the indexes it is within seconds.

Usage example:

invoices.csv

# id, whenSend, totalAmount, ...
1001, ...

invoiceLines.csv

# id, invoice_id, description, unit, qty, unit_price, amount, ...
20002, 1001, ...

./crunch \
   -in invoices.csv -indexed id \
   -in invoiceLines.csv -indexed id,invoice_id \
   -sql "SELECT invoices AS i LEFT JOIN invoiceLines AS il ON (il.invoice_id = i.id)"
   -out joined.csv

With the added indexes, such query will now execute much faster.

For now, the indexes need to be added using -indexed.

Later on, this could happen automatically for the columns appearing in JOIN, WHERE, and GROUP BY clauses.

This release is also available from Maven Central.

2.4.2: Upgrade HSQLDB to 2.7.2

Maintainance release. Micro upgrades. Kotlin upgraded to 1.8.

May not be available at the usual places under the following links (waiting if the mirroring happens).

---

Download from Sonatype OSS:
https://oss.sonatype.org/service/local/repositories/releases/content/ch/zizka/csvcruncher/csv-cruncher/2.4.2/csv-cruncher-2.4.2.jar

Or Maven Central:
https://mvnrepository.com/artifact/ch.zizka.csvcruncher/csv-cruncher

2.4.1: Fix CVE-2022-41853 in HSQLDB, upgrade HSQLDB to 2.7.1

The vulnerability allowing a malicious content to cause code execution is now addressed.

Download from Sonatype OSS:
https://oss.sonatype.org/service/local/repositories/releases/content/ch/zizka/csvcruncher/csv-cruncher/2.4.1/csv-cruncher-2.4.1.jar

Or Maven Central:
https://mvnrepository.com/artifact/ch.zizka.csvcruncher/csv-cruncher

2.4.0: Added --logLevel = ...

csv-cruncher-2.4.0

[maven-release-plugin] copy for tag csv-cruncher-2.4.0

2.3.7 - fixed module export issues on Java 12+

Fixes #87

2.3.6

• Fix: The SQL type reduction - because of how auto-cast works, real numbers were being truncated to integers.
• Fix: Handle quoted column names in 1st line of CSV.
• Fix: Quote the values in CSV converted from JSON.
• Fix: Extra line in JSON converted to CSV.
• Better handling of what formats should be left after the app run.

2.3.0

• #58 Keep the table names letter case
• Fixed JSON import conversion to CSV, and few more fixes.

2.2.0

• #59 Move all integration tests from Maven to JUnit (POM cleanup)
• Fixed #57 SELECT containing LIMIT causes SQL error
• #2 Support running a SQL script before loading the data
• #17 + #39 Quote all SQL identifiers to retain the case. feature