Skip to content

Simplify API by removing RNG interface #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Nov 25, 2020
Merged

Simplify API by removing RNG interface #8

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
1d13b15
Remove RNG interface and instead pass random number to library
aggarw13 Nov 25, 2020
91846d1
Doc updates
aggarw13 Nov 25, 2020
daab7e6
Fix CI check failures
aggarw13 Nov 25, 2020
9024eb4
Fix failing unit test
aggarw13 Nov 25, 2020
b073d6e
Add RNG seeding logic in reference example
aggarw13 Nov 25, 2020
91633b4
Seed PRNG with seconds instead of nanoseconds
aggarw13 Nov 25, 2020
14f354d
Update README example comment about RNG
aggarw13 Nov 25, 2020
99d5cd9
Update suggestion text about RNG in API header
aggarw13 Nov 25, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 24 additions & 29 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
## backOffAlgorithm Library
## backoffAlgorithm Library

This repository contains the backoffAlgorithm library, a utility library to calculate backoff period for network operation retries (like failed network connection with server) using an exponential backoff with jitter algorithm. The backoffAlgorithm library is distributed under the [MIT Open Source License](LICENSE).

Expand All @@ -22,6 +22,7 @@ The example below shows how to use the backoffAlgorithm library to retry a DNS r
#include <stdlib.h>
#include <string.h>
#include <netdb.h>
#include <time.h>

/* The maximum number of retries for the example code. */
#define RETRY_MAX_ATTEMPTS ( 5U )
Expand All @@ -32,28 +33,6 @@ The example below shows how to use the backoffAlgorithm library to retry a DNS r
/* The base back-off delay (in milliseconds) for retry configuration in the example. */
#define RETRY_BACKOFF_BASE_MS ( 500U )

/**
* A random number generator to provide to the backoffAlgorithm
* library.
*
* This function is used in the exponential backoff with jitter algorithm
* to calculate the backoff value for the next retry attempt.
*
* It is recommended to either use a True Random Number Generator (TRNG) for
* calculation of unique back-off values in devices so that collision between
* devices attempting retries at the same intervals can be avoided.
*
* For the simplicity of the code example, this function is a pseudo
* random number generator.
*
* @return The generated random number. This example function ALWAYS succeeds
* in generating a random number.
*/
static int32_t pseudoRng()
{
return( rand() % ( INT32_MAX ) );
}

int main()
{
/* Variables used in this example. */
Expand All @@ -65,6 +44,7 @@ int main()
int32_t dnsStatus = -1;
struct addrinfo hints;
struct addrinfo ** pListHead;
struct timespec tp;

/* Add hints to retrieve only TCP sockets in getaddrinfo. */
( void ) memset( &hints, 0, sizeof( hints ) );
Expand All @@ -77,10 +57,19 @@ int main()

/* Initialize reconnect attempts and interval. */
BackoffAlgorithm_InitializeParams( &retryParams,
RETRY_BACKOFF_BASE_MS,
RETRY_MAX_BACKOFF_DELAY_MS,
RETRY_MAX_ATTEMPTS,
pseudoRng );
RETRY_BACKOFF_BASE_MS,
RETRY_MAX_BACKOFF_DELAY_MS,
RETRY_MAX_ATTEMPTS );


/* Seed the pseudo random number generator used in this example (with call to
* rand() function provided by ISO C standard library) for use in backoff period
* calculation when retrying failed DNS resolution. */

/* Get current time to seed pseudo random number generator. */
( void ) clock_gettime( CLOCK_REALTIME, &tp );
/* Seed pseudo random number generator with seconds. */
srand( tp.tv_sec );

do
{
Expand All @@ -90,8 +79,14 @@ int main()
/* Retry if DNS resolution query failed. */
if( dnsStatus != 0 )
{
/* Get back-off value (in milliseconds) for the next retry. */
retryStatus = BackoffAlgorithm_GetNextBackoff( &retryParams, &nextRetryBackOff );
/* Generate a random number and get back-off value (in milliseconds) for the next retry.
* Note: It is recommended to use a random number generator that is seeded with
* device-specific entropy source so that backoff calculation in devices is different
* and possibility of network collision between devices attempting retries can be avoided.
*
* For the simplicity of the code example, the pseudo random number generator, rand() function
* is used. */
retryStatus = BackoffAlgorithm_GetNextBackoff( &retryParams, rand(), &nextRetryBackOff );
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this PRNG seeded? I ask because if you do an OTA on a fleet they will all start up with the same seed and you will have exactly the connection storm this is intended to avoid.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I have added seeding logic of RNG in the example.

}
} while( ( dnsStatus != 0 ) && ( retryStatus != BackoffAlgorithmRetriesExhausted ) );

Expand Down
19 changes: 9 additions & 10 deletions docs/doxygen/pages.dox
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ For a reference example of using the library, refer to the related README sectio

<table>
<tr>
<td colspan="4"><center><b>Code size of backoffAlgorithm library files (sizes generated with [GCC for ARM Cortex-M toolchain](https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm/downloads/9-2020-q2-update))</b></center></td>
<td colspan="4"><center><b>Code size (in bytes) of backoffAlgorithm library files (sizes generated with [GCC for ARM Cortex-M toolchain](https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm/downloads/9-2020-q2-update))</b></center></td>
</tr>
<tr>
<td><b>File</b></td>
Expand All @@ -46,9 +46,9 @@ For a reference example of using the library, refer to the related README sectio
</tr>
<tr>
<td>backoff_algorithm.c</td>
<td>0.6K</td>
<td>0.2K</td>
<td>0.2K</td>
<td>678</td>
<td>140</td>
<td>136</td>
</tr>
</table>

Expand All @@ -61,13 +61,12 @@ All functions in the backoffAlgorithm library operate only on the buffer provide
local variables on the stack.
</p>

<h3>Random Number Generator</h3>
<h3>Random Number Generation</h3>
<p>
The library requires a platform-specific random number generator for the "jitter"
part of the algorithm when generating the next backoff value. To avoid calculation
of the same random numbers across your fleet of devices attempting retry of network
operations, it is RECOMMENDED to provide a random number generator that is seeded with
an entropy source unique to the device.
The library takes a random number each time it calculates the backoff period value for the
retry attempt. To avoid calculation of the same random numbers across your fleet of devices
attempting retry of network operations, it is RECOMMENDED to generate the random number with
a random number generator that is seeded with an entropy source unique to the device.
</p>

<h3>Compliance & Coverage</h3>
Expand Down
16 changes: 9 additions & 7 deletions lexicon.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,11 @@ api
apis
aws
backoff
backoff
backoffalgorithm
backoffalgorithmretriesexhausted
backoffalgorithmrngfailure
backoffalgorithmsuccess
backoffbase
br
com
Expand All @@ -13,6 +17,7 @@ getaddrinfo
https
ifndef
inc
ingroup
iot
longjmp
maxattempts
Expand All @@ -21,20 +26,17 @@ min
mockrng
noninfringement
param
pcontext
pnextbackoff
pretrycontext
pretryparams
prng
backoffalgorithmretriesexhausted
backoffalgorithmrngfailure
backoffalgorithmsuccess
randomvalue
rng
sdk
shouldn
stderr
sublicense
tcp
utils
ingroup
backoff
pcontext
trng
utils
44 changes: 16 additions & 28 deletions source/backoff_algorithm.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,10 @@
/*-----------------------------------------------------------*/

BackoffAlgorithmStatus_t BackoffAlgorithm_GetNextBackoff( BackoffAlgorithmContext_t * pRetryContext,
uint32_t randomValue,
uint16_t * pNextBackOff )
{
BackoffAlgorithmStatus_t status = BackoffAlgorithmSuccess;
int32_t randomVal = 0;

assert( pRetryContext != NULL );
assert( pNextBackOff != NULL );
Expand All @@ -48,34 +48,24 @@ BackoffAlgorithmStatus_t BackoffAlgorithm_GetNextBackoff( BackoffAlgorithmContex
if( ( pRetryContext->attemptsDone < pRetryContext->maxRetryAttempts ) ||
( pRetryContext->maxRetryAttempts == BACKOFF_ALGORITHM_RETRY_FOREVER ) )
{
/* Generate a random number. */
randomVal = pRetryContext->pRng();
/* The next backoff value is a random value between 0 and the maximum jitter value
* for the retry attempt. */

if( randomVal < 0 )
/* Choose a random value for back-off time between 0 and the max jitter value. */
*pNextBackOff = ( uint16_t ) ( randomValue % ( pRetryContext->nextJitterMax + 1U ) );

/* Increment the retry attempt. */
pRetryContext->attemptsDone++;

/* Double the max jitter value for the next retry attempt, only
* if the new value will be less than the max backoff time value. */
if( pRetryContext->nextJitterMax < ( pRetryContext->maxBackoffDelay / 2U ) )
{
status = BackoffAlgorithmRngFailure;
pRetryContext->nextJitterMax += pRetryContext->nextJitterMax;
}
else
{
/* The next backoff value is a random value between 0 and the maximum jitter value
* for the retry attempt. */

/* Choose a random value for back-off time between 0 and the max jitter value. */
*pNextBackOff = ( uint16_t ) ( randomVal % ( pRetryContext->nextJitterMax + 1U ) );

/* Increment the retry attempt. */
pRetryContext->attemptsDone++;

/* Double the max jitter value for the next retry attempt, only
* if the new value will be less than the max backoff time value. */
if( pRetryContext->nextJitterMax < ( pRetryContext->maxBackOffDelay / 2U ) )
{
pRetryContext->nextJitterMax += pRetryContext->nextJitterMax;
}
else
{
pRetryContext->nextJitterMax = pRetryContext->maxBackOffDelay;
}
pRetryContext->nextJitterMax = pRetryContext->maxBackoffDelay;
}
}
else
Expand All @@ -94,17 +84,15 @@ BackoffAlgorithmStatus_t BackoffAlgorithm_GetNextBackoff( BackoffAlgorithmContex
void BackoffAlgorithm_InitializeParams( BackoffAlgorithmContext_t * pContext,
uint16_t backOffBase,
uint16_t maxBackOff,
uint32_t maxAttempts,
BackoffAlgorithm_RNG_t pRng )
uint32_t maxAttempts )
{
assert( pContext != NULL );

/* Initialize the context with parameters used in calculating the backoff
* value for the next retry attempt. */
pContext->nextJitterMax = backOffBase;
pContext->maxBackOffDelay = maxBackOff;
pContext->maxBackoffDelay = maxBackOff;
pContext->maxRetryAttempts = maxAttempts;
pContext->pRng = pRng;

/* The total number of retry attempts is zero at initialization. */
pContext->attemptsDone = 0;
Expand Down
39 changes: 11 additions & 28 deletions source/include/backoff_algorithm.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,28 +42,13 @@
*/
#define BACKOFF_ALGORITHM_RETRY_FOREVER 0

/**
* @brief Interface for a random number generator.
* The user should supply the platform-specific random number generator to the
* library through the @ref BackoffAlgorithm_InitializeParams API function.
*
* @note It is recommended that a true random number generator is supplied
* to the library. The random number generator should be seeded with an entropy
* source in the system.
*
* @return The random number if successful; otherwise a negative value to indicate
* failure.
*/
typedef int32_t ( * BackoffAlgorithm_RNG_t )();

/**
* @ingroup backoff_algorithm_enum_types
* @brief Status for @ref BackoffAlgorithm_GetNextBackoff.
*/
typedef enum BackoffAlgorithmStatus
{
BackoffAlgorithmSuccess = 0, /**< @brief The function successfully calculated the next back-off value. */
BackoffAlgorithmRngFailure = 1, /**< @brief The function encountered failure in generating random number. */
BackoffAlgorithmRetriesExhausted /**< @brief The function exhausted all retry attempts. */
} BackoffAlgorithmStatus_t;

Expand All @@ -77,7 +62,7 @@ typedef struct BackoffAlgorithmContext
/**
* @brief The maximum backoff delay (in milliseconds) between consecutive retry attempts.
*/
uint16_t maxBackOffDelay;
uint16_t maxBackoffDelay;

/**
* @brief The total number of retry attempts completed.
Expand All @@ -94,12 +79,6 @@ typedef struct BackoffAlgorithmContext
* @brief The maximum number of retry attempts.
*/
uint32_t maxRetryAttempts;

/**
* @brief The random number generator function used for calculating the
* backoff value for the next retry attempt.
*/
BackoffAlgorithm_RNG_t pRng;
} BackoffAlgorithmContext_t;

/**
Expand All @@ -115,15 +94,12 @@ typedef struct BackoffAlgorithmContext
* use in the exponential backoff and jitter model.
* @param[in] maxAttempts The maximum number of retry attempts. Set the value to
* #BACKOFF_ALGORITHM_RETRY_FOREVER to retry for ever.
* @param[in] pRng The platform-specific function to use for random number generation.
* The random number generator should be seeded before calling this function.
*/
/* @[define_backoffalgorithm_initializeparams] */
void BackoffAlgorithm_InitializeParams( BackoffAlgorithmContext_t * pContext,
uint16_t backOffBase,
uint16_t maxBackOff,
uint32_t maxAttempts,
BackoffAlgorithm_RNG_t pRng );
uint32_t maxAttempts );
/* @[define_backoffalgorithm_initializeparams] */

/**
Expand All @@ -135,15 +111,22 @@ void BackoffAlgorithm_InitializeParams( BackoffAlgorithmContext_t * pContext,
*
* @param[in, out] pRetryContext Structure containing parameters for the next backoff
* value calculation.
* @param[in] randomValue The random value to use for calculation of the backoff period.
* The random value should be in the range of [0, UINT32_MAX].
* @param[out] pNextBackOff This will be populated with the backoff value (in milliseconds)
* for the next retry attempt. The value does not exceed the maximum backoff delay
* configured in the context.
*
* @return #BackoffAlgorithmSuccess after a successful sleep, #BackoffAlgorithmRngFailure for a failure
* in random number generation, #BackoffAlgorithmRetriesExhausted when all attempts are exhausted.
* @note For generating a random number, it is recommended to use a Random Number Generator
* that is seeded with a device-specific entropy source so that possibility of collisions
* between multiple devices retrying the network operations can be mitigated.
*
* @return #BackoffAlgorithmSuccess after a successful sleep;
* #BackoffAlgorithmRetriesExhausted when all attempts are exhausted.
*/
/* @[define_backoffalgorithm_getnextbackoff] */
BackoffAlgorithmStatus_t BackoffAlgorithm_GetNextBackoff( BackoffAlgorithmContext_t * pRetryContext,
uint32_t randomValue,
uint16_t * pNextBackOff );
/* @[define_backoffalgorithm_getnextbackoff] */

Expand Down
Loading