fix: cache queue inventory for lower CPU usage while running queue

[levibostian]

Part of https://github.com/customerio/issues/issues/10950

This PR introduces a cache for the BQ inventory. This cache reduces the number of times the SDK needs to read the BQ inventory from the file system. This reduction in file system writes reduces the CPU usage of the SDK while the BQ runs.

todo

• QA sample apps

[github-actions]

Sample app builds 📱

Below you will find the list of the latest versions of the sample apps. It's recommended to always download the latest builds of the sample apps to accurately test the pull request.

---

• CocoaPods-FCM: levi/inmemory-inventory (1692040180)
• APN-UIKit: levi/inmemory-inventory (1692040221)

[github-actions]

Pull request title looks good 👍!

If this pull request gets merged, it will cause a new release of the software. Example: If this project's latest release version is 1.0.0. If this pull request gets merged in, the next release of this project will be 1.0.1. This pull request is not a breaking change.

All merged pull requests will eventually get deployed. But some types of pull requests will trigger a deployment (such as features and bug fixes) while some pull requests will wait to get deployed until a later time.

This project uses a special format for pull requests titles. Expand this section to learn more (expand by clicking the ᐅ symbol on the left side of this sentence)...

This project uses a special format for pull requests titles. Don't worry, it's easy!

This pull request title should be in this format:

<type>: short description of change being made

If your pull request introduces breaking changes to the code, use this format:

<type>!: short description of breaking change

where <type> is one of the following:

• feat: - A feature is being added or modified by this pull request. Use this if you made any changes to any of the features of the project.

• fix: - A bug is being fixed by this pull request. Use this if you made any fixes to bugs in the project.

• docs: - This pull request is making documentation changes, only.

• refactor: - A change was made that doesn't fix a bug or add a feature.

• test: - Adds missing tests or fixes broken tests.

• style: - Changes that do not effect the code (whitespace, linting, formatting, semi-colons, etc)

• perf: - Changes improve performance of the code.

• build: - Changes to the build system (maven, npm, gulp, etc)

• ci: - Changes to the CI build system (Travis, GitHub Actions, Circle, etc)

• chore: - Other changes to project that don't modify source code or test files.

• revert: - Reverts a previous commit that was made.

Examples:

feat: edit profile photo
refactor!: remove deprecated v1 endpoints
build: update npm dependencies
style: run formatter 

Need more examples? Want to learn more about this format? Check out the official docs.

Note: If your pull request does multiple things such as adding a feature and makes changes to the CI server and fixes some bugs then you might want to consider splitting this pull request up into multiple smaller pull requests.

[Shahroz16]

In the ideal case, it should be fine, but this can very much become a memory concern if we are hosting a large number of tasks in memory. This could easily happen if the user has a configuration configured like that or is in offline mode.

Should we add the maximum limit? but then it won't reflect the real state of the queue

[levibostian]

In early versions of the BQ, we had a cache of the inventory in-memory. The BQ was designed so that the inventory would be light-weight enough that it could be stored in memory, even if it's big. That's why the BQ uses separate files for inventory and task data, to keep the inventory light.

From my own testing, I have not noticed the inventory being too large in memory to cause concern.

I do see value in refactoring the BQ to save chunks of the BQ in the file system (chunks of the inventory and chunks of tasks). This would allow us to read and write with the file system less then we do today, even more reducing the CPU usage load. Strategies such as chunking could be considered in the future to make the SDK more performant for this scenario we are talking about here. I plan on writing a ticket to think about this performance increase.