Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Multi-process test data files generator
JavaScript
tree: 0d5c16538d

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
bin
examples
lib
test
.bob.json
.gitignore
.travis.yml
CHANGELOG.md
LICENSE
README.md
package.json

README.md

DataGen http://travis-ci.org/cliffano/datagen

Multi-process test data files generator.

This is handy when you want to generate large test data files (e.g. XMLs, JSONs, CSVs, etc), over multiple processes, utilising available CPU cores on your machine. It's also very easy to generate random numbers, dates, and strings as test data.

Installation

npm install -g datagen 

Usage

Create example header, segment, and footer template files:

datagen init

Generate 8 data files containing 1 million segments in each file, written to data1 ... data8 output files, running over 8 processes, each process is used to generate 1 file:

datagen gen -s 1000000 -w 8 -o data

Templates

DataGen uses three template files: header, segment, and footer. These templates are simple text files which are used to construct a data file in this format:

header
segment 1
segment 2
...
segment N (number of segments)
footer

Templates can contain the following parameters:

{gen_id}Unique to each datagen execution. Default value is master process PID, can be overridden via -i flag.
{worker_id}Unique to each worker. Value from 1 to number of workers.
{segment_id}Unique to each segment within the generated data file, repeated in each file. Value from 1 to number of segments. Not available in header and footer templates.
{integer()}Random integer.
{integer(100, 200)}Random integer between 100 and 200.
{float()}random float.
{float(1.23, 5.67)}Random float between 1.23 and 5.67 .
{date()}random date between 1970 and 2020 in ISO format.
{date('yyyy-mm-dd')}Random date between 1970 and 2020 in yyyy-mm-dd format.
Check out felixge/node-dateformat for more date formats.
{date(2000, 2010)}Random date between 2000 and 2010 in ISO format.
{date('yyyy-mm-dd', 2000, 2010)}Random date between 2000 and 2010 in yyyy-mm-dd format.
{select('apple', 'orange', ..., 'kiwi')}Select one item from the arguments. You can have as many arguments as you want.
{word()}Random word from Lorem Ipsum.
{word(3)}Random 3 words from Lorem Ipsum.
{first_name()}Random first name.
{last_name()}Random last name.
{email()}Random email address.
{phone()}Random phone number. Default format #### ####.
{phone('(###) ####-####')}Random phone number with a custom format. Each '#' will be replaced by a random number.
Something went wrong with that request. Please try again.