Skip to content

Fix flaky test #1158

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 14, 2020
Merged

Fix flaky test #1158

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8c610be
Fix flaky test
delvedor Apr 14, 2020
5b5f307
Fix #1154
delvedor Apr 14, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions test/behavior/resurrect.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,7 @@ test('Should execute the recurrect API with the ping strategy', t => {
})

q.add((q, done) => {
cluster.kill('node0')
setTimeout(done, 100)
cluster.kill('node0', done)
})

q.add((q, done) => {
Expand Down Expand Up @@ -173,8 +172,7 @@ test('Should execute the recurrect API with the optimistic strategy', t => {
})

q.add((q, done) => {
cluster.kill('node0')
setTimeout(done, 100)
cluster.kill('node0', done)
})

q.add((q, done) => {
Expand Down
54 changes: 43 additions & 11 deletions test/behavior/sniff.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@

const { test } = require('tap')
const { URL } = require('url')
const lolex = require('lolex')
const workq = require('workq')
const { buildCluster } = require('../utils')
const { Client, Connection, Transport, events, errors } = require('../../index')

Expand Down Expand Up @@ -111,8 +113,10 @@ test('Should handle hostnames in publish_address', t => {
})
})

test('Sniff interval', { skip: 'Flaky on CI' }, t => {
t.plan(10)
test('Sniff interval', t => {
t.plan(11)
const clock = lolex.install({ toFake: ['Date'] })
const q = workq()

buildCluster(({ nodes, shutdown, kill }) => {
const client = new Client({
Expand All @@ -132,19 +136,47 @@ test('Sniff interval', { skip: 'Flaky on CI' }, t => {
})

t.strictEqual(client.connectionPool.size, 1)
setTimeout(() => client.info(t.error), 60)

setTimeout(() => {
// let's kill a node
kill('node1')
client.info(t.error)
}, 150)
q.add((q, done) => {
clock.tick(51)
client.info(err => {
t.error(err)
waitSniffEnd(() => {
t.strictEqual(client.connectionPool.size, 4)
done()
})
})
})

q.add((q, done) => {
kill('node1', done)
})

setTimeout(() => {
t.strictEqual(client.connectionPool.size, 3)
}, 200)
q.add((q, done) => {
clock.tick(51)
client.info(err => {
t.error(err)
waitSniffEnd(() => {
t.strictEqual(client.connectionPool.size, 3)
done()
})
})
})

t.teardown(shutdown)

// it can happen that the sniff operation resolves
// after the API call that trioggered it, so to
// be sure that we are checking the connectionPool size
// at the right moment, we verify that the transport
// is no longer sniffing
function waitSniffEnd (callback) {
if (client.transport._isSniffing) {
setTimeout(waitSniffEnd, 500, callback)
} else {
callback()
}
}
})
Comment on lines +173 to 180
Copy link

@penge penge Apr 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't know about client.transport._isSniffing Seems like a nice trick. I was dealing with the same, so what I did, I just ignored the last sniff. Because even when expecting it to happen 3-times (2 nodes killed) or 2-times (1 node killed), it had sometimes a chance to be called one more time.
500 seems like enough of waiting time

Copy link
Member Author

@delvedor delvedor Apr 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

500 seems like enough of waiting time

500 is just a random number, it could also be 50, the point here is that I'm not waiting for 500 and then doing something, but I'm checking a value that represents what I need every 500ms before starting the next operation. In this way the test code is more reliable :)

Copy link

@penge penge Apr 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got lost at random number :)

})

Expand Down
26 changes: 8 additions & 18 deletions test/unit/transport.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -578,9 +578,8 @@ test('Retry mechanism', t => {
if (count > 0) {
res.end(JSON.stringify({ hello: 'world' }))
} else {
setTimeout(() => {
res.end(JSON.stringify({ hello: 'world' }))
}, 1000)
res.statusCode = 504
res.end(JSON.stringify({ error: true }))
}
count++
}
Expand All @@ -603,7 +602,6 @@ test('Retry mechanism', t => {
connectionPool: pool,
serializer: new Serializer(),
maxRetries: 1,
requestTimeout: 250,
sniffInterval: false,
sniffOnStart: false
})
Expand All @@ -624,15 +622,10 @@ test('Should not retry if the body is a stream', t => {

var count = 0
function handler (req, res) {
res.setHeader('Content-Type', 'application/json;utf=8')
if (count > 0) {
res.end(JSON.stringify({ hello: 'world' }))
} else {
setTimeout(() => {
res.end(JSON.stringify({ hello: 'world' }))
}, 1000)
}
count++
res.setHeader('Content-Type', 'application/json;utf=8')
res.statusCode = 504
res.end(JSON.stringify({ error: true }))
}

buildServer(handler, ({ port }, server) => {
Expand All @@ -653,7 +646,6 @@ test('Should not retry if the body is a stream', t => {
connectionPool: pool,
serializer: new Serializer(),
maxRetries: 1,
requestTimeout: 10,
sniffInterval: false,
sniffOnStart: false
})
Expand All @@ -663,7 +655,7 @@ test('Should not retry if the body is a stream', t => {
path: '/hello',
body: intoStream(JSON.stringify({ hello: 'world' }))
}, (err, { body }) => {
t.ok(err instanceof TimeoutError)
t.ok(err instanceof ResponseError)
t.strictEqual(count, 1)
server.stop()
})
Expand All @@ -679,9 +671,8 @@ test('Custom retry mechanism', t => {
if (count > 0) {
res.end(JSON.stringify({ hello: 'world' }))
} else {
setTimeout(() => {
res.end(JSON.stringify({ hello: 'world' }))
}, 1000)
res.statusCode = 504
res.end(JSON.stringify({ error: true }))
}
count++
}
Expand All @@ -704,7 +695,6 @@ test('Custom retry mechanism', t => {
connectionPool: pool,
serializer: new Serializer(),
maxRetries: 0,
requestTimeout: 250,
sniffInterval: false,
sniffOnStart: false
})
Expand Down
9 changes: 6 additions & 3 deletions test/utils/buildCluster.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,14 +56,17 @@ function buildCluster (options, callback) {

function shutdown () {
debug(`Shutting down cluster '${clusterId}'`)
Object.keys(nodes).forEach(kill)
for (const id in nodes) {
kill(id)
}
}

function kill (id) {
function kill (id, callback) {
debug(`Shutting down cluster node '${id}' (cluster id: '${clusterId}')`)
nodes[id].server.stop()
const node = nodes[id]
delete nodes[id]
delete sniffResult.nodes[id]
node.server.stop(callback)
Copy link

@penge penge Apr 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized later that sniff result endpoint is mocked for the tests. So if I waited or didn't wait, I think I was getting the same results. But I think in case of resurrect it is needed to wait actually.

}

function spawn (id, callback) {
Expand Down