Skip to content
This repository has been archived by the owner on Dec 8, 2022. It is now read-only.

improve defender unit tests #258

Merged
merged 2 commits into from
Jan 30, 2019

Conversation

qiutongs
Copy link
Contributor

Description

  • not do test assert in MQTT callback; instead, copy the data and assert outside the callback
  • remove "retry if reject" logic; instead, simply increase the sleep time

Checklist:

  • I have tested my changes. No regression in existing tests.
  • My code is Linted.

@@ -130,10 +133,13 @@ TEST_TEAR_DOWN( Full_DEFENDER )
{
AwsIotDefender_Stop();

if( _reportAccepted || _reportRejected )
/* Actually get defender callback. */
if( _callbackInfo.eventType != -1 )
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chatted offline. It seems in this implementation, we leave throttling handling to user. To be specific, application is increasing _DEFENDER_PUBLISH_INTERVAL_SECONDS, and parsing _callbackInfo.eventType to determine what to do next.

This may not be the best solution. Since:

  1. This creates unreliable test. Throttling is usually per region per account, a fix number with single attempt would never reliably work.
  2. Inconsistent with other AWS clients. Where usually in other clients, users can input retry scheme, and max retry. (e.g. typical retry scheme -- exponential backoff delay. typical max retry -- 3.)

This review comment is more about design instead of this specific PR. Apologies.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with #1. And I don't have a good solution now.

#2 will not be a problem for actual user. Defender is a daemon application that will send metrics as slow as 5 minutes (unless user keeps start, stop, start, stop...)

So this problem is scoped to our testing.

_reportRejected = pCallbackInfo->eventType == AWS_IOT_DEFENDER_METRICS_REJECTED;
if( _callbackInfo.payloadLength > 0 )
{
_callbackInfo.pPayload = AwsIotTest_Malloc( _callbackInfo.payloadLength );
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line 503, 512, 533, and 534. Symmetry is beauty. Better to put allocate/free both in test setup/teardown. Or both in local test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense. I can move the allocate to setup

Copy link
Member

@yuhui-zheng yuhui-zheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generic comments provided. Approving as a non content expert.

@qiutongs qiutongs merged commit 2244fe5 into feature/ble-beta Jan 30, 2019
@qiutongs qiutongs deleted the qiutongs/defender_tests_improve branch January 30, 2019 19:24
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants