Patching an identity provider with a large number of applications takes too long

[johnmaia]

Patching an identity provider with a large number of applications takes too long

Description

When patching an identity provider to activate it on a particular application and its already activated on a large number of applications the request takes too long to complete.

We've encountered this issue our development environment where we have 2K+ applications that already have Google IdP activated and every time we activate it on a new application the call takes about 20 seconds to complete.

This will potentially lead to timeout from our service given that we are activating Google and Facebook IdP every time a new application is registered and its already taking 40 seconds.

I understand that FusionAuth may not been originally design to handle with a large number of applications, but this is an issue of scalability that should be taken into account.

Detail

Some of this has been addressed in prior work, but we are still loading up all applications into the application tab in the IDP edit page in the admin UI, what remains is still updating the application tab so that we do the same normalized query that we do on the tenant tab and then optionally decide if we are going to clean up redundant config in the database.

Related

• Pagination for everyone! And all things! #374
• Identity Provider API PATCH requests fail if request includes applicationConfiguration #2454
• Pagination/Search for Identity Providers Application Options #2657

[robotdan]

Thanks @johnmaia yes - this is something we want to enhance in the near future. When we deliver Issue #374 we plan to review a bunch of code paths that may be currently negatively affected by a large number of applications.

[robotdan]

@johnmaia is this still slow on recent versions of FusionAuth?

[robotdan]

Internal:
We should plan to use same strategy we have for the tenant config tab on an IdP for applications. This means we'll need an AJAX search action and a Handlebars template to just add config for specific applications. This will limit the size of DOM on this page.

• Either through migration, or next touch, we should normalize the application config for an IdP. This will prune out default configurations that are bloating the config.
• Update the UI to allow for explicit application config, instead of just a list of applications.
• Review API, I don't think this will require any changes.

[spwitt]

Daniel's proposed solution would improve UI performance if the IdP was only enabled for a small number of applications. The most common use case is likely to be the case that John mentioned in the issue description where the IdP is enabled for most (or all) applications and needs to be added for a new application.

Resolving the performance issue in the admin portal will require new UI patterns. For now, we will confirm that the PATCH API functions properly with regards to adding the application config for a new application. This will provide a performant option to update the IdP configuration with a large number of applications.

[spwitt]

Updating the admin UI to be more performant in cases where a large number of applications exist is a larger task than expected. We will be fixing a bug with the PATCH /api/identity-provider API to allow modifying the configuration for a single application until we have time to properly address UI performance issues. This will provide a way to add configuration for a single application without loading the admin UI or performing a full GET/PUT for the entire IdP.

• Identity Provider API PATCH requests fail if request includes applicationConfiguration #2454