Skip to content

Commit

Permalink
feat BUG-2712 Adding reproducer for memory leak bug
Browse files Browse the repository at this point in the history
  • Loading branch information
william-wallace-simplisafe committed Jul 14, 2022
1 parent a5018ec commit d31735d
Show file tree
Hide file tree
Showing 8 changed files with 2,313 additions and 0 deletions.
4 changes: 4 additions & 0 deletions test/2712_reproducer/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
FROM node:16-alpine
WORKDIR /usr/app
COPY . .
RUN npm install
41 changes: 41 additions & 0 deletions test/2712_reproducer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Bug 2712 Reproducer
This bug happens while using newman as a library with a data file containing multiple iterations.

## About this reproducer
This reproducer uses docker-compose to run a node.js script which uses newman library so it can write to a file (failures.csv) The `echoRunner.postman_collection.json` hits the `https://postman-echo.com/post` endpoint once for each iteration using the data provided in the datafile. Failures are contrived mathmatically to produce some failures which will be written to the failures.csv file.

## Running this reproducer
* Prerequisites: docker-compose, docker
To run a quick sanity check with a data file with one iteration do the following
```
export DATAFILE=data/one.csv; docker-compose up --build
```

To reproduce the bug, set the `$DATAFILE` env variable to the `uids.csv` containing multple iterations. This reproducer will run out of heap space at approximately iteration 32200

```
export DATAFILE=data/uids.csv; docker-compose up --build
```
```
. . .
newman_1 | Iteration 32173/121209
newman_1 |
newman_1 | → echo
newman_1 | POST https://postman-echo.com/post
newman_1 | SUCCESS: uid: 315769, size: undefined
newman_1 | [200 OK, 1.02kB, 25ms]
newman_1 | ✓ Status code is 200
newman_1 | ✓ Check uid % 50 > 7
newman_1 |
newman_1 | <--- Last few GCs --->
newman_1 |
newman_1 | [1:0x7fa9d36693b0] 1605430 ms: Mark-sweep 981.7 (1058.0) -> 968.6 (1058.3) MB, 691.6 / 0.1 ms (average mu = 0.369, current mu = 0.348) task scavenge might not succeed
newman_1 | [1:0x7fa9d36693b0] 1606491 ms: Mark-sweep 982.0 (1058.3) -> 968.8 (1058.5) MB, 652.4 / 0.1 ms (average mu = 0.377, current mu = 0.385) task scavenge might not succeed
newman_1 | [1:0x7fa9d36693b0] 1607515 ms: Mark-sweep 982.2 (1058.5) -> 969.0 (1058.8) MB, 632.9 / 0.1 ms (average mu = 0.379, current mu = 0.382) task scavenge might not succeed
newman_1 |
newman_1 |
newman_1 | <--- JS stacktrace --->
newman_1 |
newman_1 | FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
echorunner_newman_1 exited with code 139
```
4 changes: 4 additions & 0 deletions test/2712_reproducer/data/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Ignore everything in this directory
*
# Except this file
!.gitignore
16 changes: 16 additions & 0 deletions test/2712_reproducer/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
version: "3"
services:
newman:
build: .
volumes:
- .:/usr/app/
- /usr/app/node_modules
environment:
- NODE_OPTIONS=--max-old-space-size=1024
- COLLECTION=${COLLECTION}
- ENV=${ENV}
- DATAFILE=${DATAFILE}
- ZUORA_CLIENT_ID=${ZUORA_CLIENT_ID}
- ZUORA_CLIENT_SECRET=${ZUORA_CLIENT_SECRET}
- LOG_LEVEL=${LOG_LEVEL}
command: node index.js -e $ENV -c $COLLECTION -d $DATAFILE
128 changes: 128 additions & 0 deletions test/2712_reproducer/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
const newman = require('newman');
const yargs = require('yargs');
const figlet = require('figlet');
const fs = require('fs');

const failuresFile = 'failures.csv';
const writer = fs.createWriteStream(failuresFile);
var collection = 'postman/echoRunner.postman_collection.json';
var env = '';
var data = '';
const myRequest = 'echo';


const environmentVars = [];
const globalVars = [];
const failures = [];

figlet("echoRunner", function (err, banner) {
if (err) {
console.log('Something went wrong . . .');
console.dir(err);
return;
}
console.log(banner);
});

yargs.version('0.0.1');
const argv = yargs
.option('env', {
alias: 'e',
description: 'Specify the postman environment.json file to use',
type: 'string'
})
.option('collection', {
alias: 'c',
description: 'Specify the postman collection file to use',
type: 'string'
})
.option('data', {
alias: 'd',
description: 'Specify the postman iteration-data.json file to use',
type: 'string'
})
.help().alias('help','h').argv;

if (argv.env) {
env = argv.env;
}
if (argv.collection) {
collection = argv.collection;
}
if (argv['data']) {
data = argv['data'];
}
if (argv.version) {
yargs.showVersion('log');
}

console.log(`Using '${collection}' for collection`);
console.log(`Using '${env}' for environment`);
console.log(`Using '${data}' for iteration-data`);

const parseUid = (rawBody) => {
const query = rawBody.replace('\\','');
const queryStringObj = JSON.parse(query);
const queryString = queryStringObj.queryString;
const equalsIndex = queryString.indexOf('=');
const start = queryString.indexOf('\'',equalsIndex)+1;
const end = queryString.indexOf('\'', start);
const myuid = queryString.substring(start, end).trim();
return myuid;
};




writer.write('uid\n');
newman.run({
collection: collection,
iterationData: data,
environment: env,
envVar: environmentVars,
globalVar: globalVars,
reporters: 'cli'
}, function (err) {
if (err) { throw err; }
console.log('collection run complete!');
})
.on('request', function (error, args) {
if (args.item.name === myRequest) {
if (error) {
console.error(error);
}
else {
var rawBody = args.response.stream,
body = rawBody.toString();
try {

const rawBody = args.request.body.raw
const uid = parseUid(rawBody);

const json = JSON.parse(body);
console.log() // bug: missing newline in previous output
const size = json.length;
if (Number(uid) % 50 <= 7) {
console.log(`FAILED: uid: ${uid}`)
failures.push(uid);
writer.write(`${uid}\n`);
} else {
console.log(`SUCCESS: uid: ${uid}, size: ${size}`);
}
} catch (e) {
console.error("Response was not json");
console.error(e)
}
}
}
})
.on('done', function (err, summary) {
var jsonResults = JSON.stringify(failures, null, 4)
// fs.writeFileSync('failures.json', jsonResults);
// console.log(`failures: ${jsonResults}`);
if (failures.length) {
process.exit(1);
} else {
process.exit(0);
}
});

0 comments on commit d31735d

Please sign in to comment.