ugly_csv_generator

Python package to automatically uglify CSVs. Why? To improve the testing capabilities of pipelines that must be able to support strongly malformed input data.

All the malformation automated here are non-destructive, meaning they introduce confusion in the data but do not any information.

The inspiration for the automated malformation are all from real-life CSVs (sigh)

Humans will always surprise us with the ever-new malformed input data, but hey, we can try to best ruining the test CSVs!

How do I install this package?

As usual, just download it using pip:

pip install ugly_csv_generator

Tests Coverage

Since some software handling coverages sometime get slightly different results, here's three of them:

Python package to generate ugly real-looking csvs.

Usage example

To ruin a CSV you can use the following snippet. In the following example we use a tool to generate a random sanitized csv.

from random_csv_generator import random_csv
from ugly_csv_generator import uglify

csv = random_csv(5) # CSV with 5 lines
csv = csv[csv.columns[:3]] # We will use only the first 3 columns for this example
ugly = uglify(csv)

The initial CSV will look something like:

region	province	surname
Puglia	Lecce	Righetti
Campania	Napoli	Govoni
Emilia Romagna	Reggio Emilia	Vichi
Lombardia	Lecco	Costa
Umbria	Perugia	Fabbro

The result uglified CSV will look something like this:

0	1	2	3	4	5
nan	nan	nan	nan	nan	nan
region-1	region	region 0	province	surname	nan
nan	nan	nan	nan	nan	nan
nan	nan	nan	nan	nan	nan
nan	nan	nan	nan	nan	nan
nan	region	nan	province	surname	nan
nan	region	nan	province	surname	nan
nan	Puglia	nan	Lecce	Righetti	nan
nan	Campania	nan	Napoli	Govoni	nan
nan	Emilia Romagna	nan	Reggio Emilia	Vichi	nan
nan	Lombardia	nan	Lecco	Costa	nan
nan	Umbria	nan	Perugia	Fabbro	nan
nan	nan	nan	nan	nan	nan
nan	nan	nan	nan	nan	nan

The uglify method offers numerous keyword parameters, since the library is currently in quick evolution if you are interested in them just check out the code documentation.