Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GeoIP performance: database expiration and InetAddress optimization #4286

Conversation

dlvenable
Copy link
Member

Description

From performance testing of the geoip processor I found two unrelated areas of unnecessary performance slowdowns.

Database expiration

The geoip processor was check if the database was expired for each event in a batch. This is a surprisingly slow operation because Instant.now() calls VM.getNanoTimeAdjustment(localOffset) which is somewhat slow.

This PR helps by adding a BatchGeoIPDatabaseReader which holds the value of isExpired for the whole batch. We don't re-evaluate for each event.

InetAddress optimization

Parsing the IP address is somewhat slow. The geoip processor was doing this twice. I consolidated this work into a single method GeoInetAddress::usableInetFromString which parses the string and then returns a non-empty Optional<InetAddress> if this IP address is usable (currently if it is public or not). Some of the refactoring here is also to make it more generic.

Finally, I moved the main processing into its own method to help with readability.

Issues Resolved

N/A

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…ddress once and track the database expiration for each batch of the geoip processor.

Signed-off-by: David Venable <dlv@amazon.com>
Signed-off-by: David Venable <dlv@amazon.com>
@dlvenable dlvenable force-pushed the geoip-expiry-inet-performance-improvements branch from 90d7b52 to 581e53b Compare March 15, 2024 15:56
@dlvenable dlvenable merged commit d0f8ab9 into opensearch-project:main Mar 19, 2024
46 of 47 checks passed
@dlvenable dlvenable deleted the geoip-expiry-inet-performance-improvements branch March 19, 2024 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants