Import API can cause the elastic index refresh_interval to be set to -1s unintentionally

[jobannon]

Concurrent Import API can cause the elastic index refresh_interval to be set to -1s unintentionally

Description

If two imports are completed at the roughly same time, a race condition exists where the refresh_internal might be set to -1 after the import is complete. Once an import completes, the refresh interval should be 1s and our default setting for the refresh interval is 1s. When the refresh_interval is -1, indexed data (well since the last time that ES has updated its cache) is not available in searches.

Observed versions

1.48.3

Affects versions

Steps to reproduce

Steps to reproduce the behavior:

1. Run an import of users ("batch one")
2. Run a second import shortly after the first ("batch two")
3. Make sure that the second batch starts before the first batch completes, and that the second batch finishes after the first batch completes.
4. Observe that the refresh interval is not returned to 1s after the import process completes.

Expected behavior

For performance reasons, we increase/turn off the refresh_interval during an import. However, we should always return this interval to a default like 1s.

Screenshots

If applicable, add screenshots to help explain your problem. Delete this section if it is not applicable.

Platform

FA Dev setup

Community guidelines

All issues filed in this repository must abide by the FusionAuth community guidelines.

Additional context

The refresh interval can be checked using the following command

curl -X GET "http://localhost:9021/fusionauth_user/_settings"

The refresh interval can be updated with the following command (must have access to ES service)

curl --request PUT \
  --url http://localhost:9021/fusionauth_user/_settings \
  --header 'Content-Type: application/json' \
  --data '{
        "index": {
                "refresh_interval": "1s"
        }
}

'

Documentation

• Update documentation for fusionauth-app.search.default-refresh-interval config value

Release Notes

Fixes a race condition in the bulk user import API /api/user/import that could set the Elasticsearch refresh_interval to -1 and prevent newly created users from being searchable. This condition was triggered by concurrent bulk requests with specific timing.

[robotdan]

Internal:

• https://github.com/FusionAuth/fusionauth-app/pull/397
• document fusionauth-app.search.default-refresh-interval. and fix links fusionauth-site#2974