Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

services/horizon: added indexes by id for cb and liquidity pool on tx/ops tables #4477

Merged
merged 3 commits into from
Jul 27, 2022

Conversation

sreuland
Copy link
Contributor

@sreuland sreuland commented Jul 25, 2022

PR Checklist

PR Structure

  • This PR has reasonably narrow scope (if not, break it down into smaller PRs).
  • This PR avoids mixing refactoring changes with feature changes (split into two PRs
    otherwise).
  • This PR's title starts with name of package that is most changed in the PR, ex.
    services/friendbot, or all or doc if the changes are broad or impact many
    packages.

Thoroughness

  • This PR adds tests for the most critical parts of the new functionality or fixes.
  • I've updated any docs (developer docs, .md
    files, etc... affected by this change). Take a look in the docs folder for a given service,
    like this one.

Release planning

  • I've updated the relevant CHANGELOG (here for Horizon) if
    needed with deprecations, added features, breaking changes, and DB schema changes.
  • I've decided if this PR requires a new major/minor version according to
    semver, or if it's mainly a patch change. The PR is targeted at the next
    release branch if it's not a patch change.

What

added indexes on the cb and liquidity pool id columns in their respective tx/ops materialized join tables

Why

there were slow query performance happening after growth of these tables, and some queries were joining on the id itself which wasn't indexed yet, waiting on long table scans.
Closes #4455
Closes #4436

Known limitations

N/A

@sreuland sreuland self-assigned this Jul 25, 2022
@sreuland
Copy link
Contributor Author

I looked into a few other candidates, history_assets table didn't see need for further indexing, and accounts via history_transaction_participants and history_operation_participants, those tables have a compound index that starts with history_account_id, so, that index can be pulled by query plan in queries by account id.

@sreuland sreuland requested a review from a team July 25, 2022 19:26
Copy link
Contributor

@bartekn bartekn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Do you have any performance stats for this change? Not required, just wonder what's the speed improvements in prd vs stg with mirroring turned on.

@sreuland
Copy link
Contributor Author

@bartekn , yes, from report on #4455 , /claimable_balances/{cb_id}/transactions, tested on stg, saw response times go from 10+ seconds to sub-second with the indexes in place.

@sreuland sreuland merged commit c2d52f0 into stellar:master Jul 27, 2022
@2opremio
Copy link
Contributor

Any idea of how much do the indices occupy?

@sreuland
Copy link
Contributor Author

Any idea of how much do the indices occupy?

each index created on stg(with pubnet history from genesis), took about 6GB of size.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants