[CoE Starter Kit - BUG] ISSUE Full vs Incremental Inventory logic causing troubles in big tenants

[miroslav-harlas]

Describe the issue

We have a big tenant and a full inventory is taking about 9 days as of now in our tenant (incremental fortunately less than 1 day). When we installed the CoE Starter Kit, we modified the schedule of Admin | Sync Template v3 to avoid parallel inventory runs . First inventory run failed, and after some some time we triggered the second of full inventory. When second run was completed, I was on vacation and came back after two weeks to start the incremental inventory, however we modified the schedule of Admin | Sync Template v3 to have it triggered once a week only. Later on we realized that we are missing a number of apps and flows from our tenant in the CoE starter Kit inventory and the root cause is the logic of incremental inventory.

We identified that there is action named Get past time inside of the Admin | Sync Template v3 (Flows) and in some others, which is defining the incremental as inventory of those objects which were created in last 1 week. Because we played with a schedule - and that was due to the fact that we have a big tenant - that caused that a number of apps and flows was missed and it can be fixed only by running with full Inventory.

The follow-up problem is:
When we run full inventory - that is running for about 9 days. During full inventory, we are preventing CoE starter Kit form not running incremental. When full inventory is finished and we run incremental, the incremental run do not take all apps and flows since the start of previous full inventory, because and is causing missing data in CoE starter Kit. There are plenty of makers in our tenant, who creates apps and flows on daily basis and there is no out-of-the-box safe way how to have complete data.

Our current workaround is, that after full inventory is completed, we update the Get past time calculations in all imapcted cloud flows, e.g. to include data from last two or three weeks, to ensure that nothing is missing.

Expected Behavior

Implement new environment variable, which will be defining a number of days for calculating the incremental. The default value can be for example 7 days.

If this solution is implemented, it gives admins the option to play with schedule of Admin | Sync Template v3, which may be beneficial not only for big tenants. In case of small tenants, where incremental is running daily without troubles, PP CoE admins can change the value to just 2 days, which may increase a performance as well. In case that something goes wrong and CoE inventory components stop to work longer time than week (due to whatever issue, e.g. unsuccessful update of the solution to new version, etc...), after such an issue CoE Admins do not have to trigger full inventory, they can just increase the interval for incremental inventory to needed number (e.g. 14 days) for temporary time, and later on reduce it back again to smaller number.

What solution are you experiencing the issue with?

Core

What solution version are you using?

3.40

What app or flow are you having the issue with?

Admin | Sync Template v3 (Flows)

Steps To Reproduce

No response

Anything else?

Looking forward to have BYODL integrated with CoE starter Kit. We are excited to here more from you once this feature is available.

[Jenefer-Monroe]

Thats an easy fix I can target for Oct. Thanks for posting!

[Jenefer-Monroe]

Added env var for Oct, and applied to app and flow sync flows.

[CoEStarterKitBot]

@miroslav-harlas This has been fixed in the latest release. Please install the latest version of the toolkit following the instructions for installing updates. Note that if you do not remove the unmanaged layers as described there you will not receive updates from us.