Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse code

Worker ID and segment ID starts from 1 instead of 0.

  • Loading branch information...
commit c4d9dbb35870b1cee7f1983a0a03cf6eeb6446ff 1 parent e437ba6
Cliffano Subagio authored
1  CHANGELOG.md
Source Rendered
... ... @@ -1,5 +1,6 @@
1 1 ### 0.0.3 (SNAPSHOT)
2 2 * Worker write no longer streams non-stop, better handling of fs backpressure
  3 +* Worker ID and segment ID starts from 1 instead of 0
3 4
4 5 ### 0.0.2
5 6 * Rename run command to gen
8 README.md
Source Rendered
@@ -27,18 +27,18 @@ Templates
27 27 DataGen uses three template files: header, segment, and footer. These templates are simple text files which are used to construct a data file in this format:
28 28
29 29 header
30   - segment 0
31 30 segment 1
  31 + segment 2
32 32 ...
33   - segment N (number of segments - 1)
  33 + segment N (number of segments)
34 34 footer
35 35
36 36 Templates can contain the following parameters:
37 37
38 38 <table>
39 39 <tr><td>{gen_id}</td><td>Unique to each datagen execution. Default value is master process PID, can be overridden via -i flag.</td></tr>
40   -<tr><td>{worker_id}</td><td>Unique to each worker. Value from 0 to number of workers - 1.</td></tr>
41   -<tr><td>{segment_id}</td><td>Unique to each segment within the generated data file, repeated in each file. Value from 0 to number of segments - 1. Not available in header and footer templates.</td></tr>
  40 +<tr><td>{worker_id}</td><td>Unique to each worker. Value from 1 to number of workers.</td></tr>
  41 +<tr><td>{segment_id}</td><td>Unique to each segment within the generated data file, repeated in each file. Value from 1 to number of segments. Not available in header and footer templates.</td></tr>
42 42 <tr><td>{integer()}</td><td>Random integer.</td></tr>
43 43 <tr><td>{integer(100, 200)}</td><td>Random integer between 100 and 200.</td></tr>
44 44 <tr><td>{float()}</td><td>random float.</td></tr>
2  lib/datagen.js
@@ -45,7 +45,7 @@ DataGen.prototype.generate = function (genId, numSegments, numWorkers, outFile)
45 45 for (var i = 0; i < numWorkers; i += 1) {
46 46 var worker = childProcess.fork(p.join(__dirname, 'worker.js'));
47 47 worker.send({
48   - workerId: i,
  48 + workerId: i + 1,
49 49 templates: templates,
50 50 genId: genId,
51 51 numSegments: numSegments,
4 test/datagen.js
@@ -68,7 +68,7 @@ describe('datagen', function () {
68 68 // only 1 worker by default
69 69 checks.child_process_fork_sends.length.should.equal(1);
70 70
71   - checks.child_process_fork_sends[0].workerId.should.equal(0);
  71 + checks.child_process_fork_sends[0].workerId.should.equal(1);
72 72 checks.child_process_fork_sends[0].genId.should.equal('56789');
73 73 checks.child_process_fork_sends[0].numSegments.should.equal(1);
74 74 checks.child_process_fork_sends[0].outFile.should.equal('data');
@@ -81,7 +81,7 @@ describe('datagen', function () {
81 81 // one message for each worker
82 82 checks.child_process_fork_sends.length.should.equal(8);
83 83
84   - checks.child_process_fork_sends[0].workerId.should.equal(0);
  84 + checks.child_process_fork_sends[0].workerId.should.equal(1);
85 85 checks.child_process_fork_sends[0].genId.should.equal('12345');
86 86 checks.child_process_fork_sends[0].numSegments.should.equal(10000);
87 87 checks.child_process_fork_sends[0].outFile.should.equal('somedatafile');

0 comments on commit c4d9dbb

Please sign in to comment.
Something went wrong with that request. Please try again.