Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: use better queries for listLogs endpoint #2014

Merged
merged 3 commits into from May 29, 2019

Conversation

zachdaniel
Copy link
Contributor

@zachdaniel zachdaniel commented May 23, 2019

#1858

Motivation

  • The listLogs api endpoint is very slow for moderate-large block ranges. The queries that this PR replaces would take about 500 seconds. These ones still aren't that fast, but they complete in 15-30 seconds for those same block ranges so it is a vast improvement.

Changelog

Bug Fixes

  • Use a more optimized query when provided with an address hash filter. We first limit our search to logs from transactions and internal transactions that were from, to, or creating the provided address. This limits the search space significantly.

    • I added an entry to CHANGELOG.md with this PR
    • If I added new functionality, I added tests covering it.
    • If I fixed a bug, I added a regression test to prevent the bug from silently reappearing again.
    • I checked whether I should update the docs and did so if necessary

Note: The existing tests reinforce correct behavior, but I can't write a unit test to confirm the performance characteristics.

@coveralls
Copy link

coveralls commented May 23, 2019

Pull Request Test Coverage Report for Build f0131f74-43d6-4030-bfe0-8af47db002df

  • 19 of 19 (100.0%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.04%) to 80.912%

Totals Coverage Status
Change from base Build aedde6f0-b477-4623-8632-93b5d7675241: 0.04%
Covered Lines: 4773
Relevant Lines: 5899

💛 - Coveralls

@zachdaniel zachdaniel force-pushed the list-logs-endpoint-performance branch 2 times, most recently from b453689 to 3011d65 Compare May 23, 2019 16:02
vbaranov pushed a commit that referenced this pull request May 24, 2019
Copy link
Member

@vbaranov vbaranov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zachdaniel

Could you please provide a live example to compare responses between prod/stg where this update gives benefit? Because of an example from the original issue still returns me 504 error on stg.

Copy link
Contributor

@pasqu4le pasqu4le left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I only have a couple of minor comments

apps/explorer/lib/explorer/etherscan/logs.ex Outdated Show resolved Hide resolved
apps/explorer/lib/explorer/etherscan/logs.ex Outdated Show resolved Hide resolved
apps/explorer/lib/explorer/etherscan/logs.ex Outdated Show resolved Hide resolved
@zachdaniel zachdaniel force-pushed the list-logs-endpoint-performance branch from cf323f7 to c39b39b Compare May 28, 2019 17:40
@zachdaniel zachdaniel force-pushed the list-logs-endpoint-performance branch from c39b39b to 2ccdfab Compare May 28, 2019 18:02
@vbaranov vbaranov self-requested a review May 29, 2019 19:23
Copy link
Member

@vbaranov vbaranov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works much faster on stg 👍

@vbaranov vbaranov merged commit 56440ed into master May 29, 2019
@vbaranov vbaranov deleted the list-logs-endpoint-performance branch May 29, 2019 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants