New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create an index on the submissions table #78

Merged
merged 1 commit into from Dec 22, 2018

Conversation

Projects
None yet
3 participants
@kmyk
Copy link
Contributor

kmyk commented Dec 22, 2018

とりあえず最優先で張るべきぽいindexを張ります。
最大のボトルネックであるだろう https://kenkoooo.com/atcoder/atcoder-api/results?user=kimiyuki&rivals=kmjp のendpointがこちらの鯖の実測で2倍速ぐらい、内部のSQLの速度は100倍速ぐらいになります。

Scalaのコードだと次の関数:

def loadUserSubmissions(userIds: UserId*): Iterator[Submission] = {
SubmissionIterator(
this,
selectFrom(Submission as SubmissionSyntax).where.in(SubmissionSyntax.userId, userIds)
)
}

速度 張る前:

atcoder_problems=> DROP INDEX submissions_user_id_idx;
DROP INDEX
atcoder_problems=> EXPLAIN ANALYZE SELECT * FROM submissions WHERE user_id IN ('kimiyuki', 'kenkoooo');
                                                           QUERY PLAN                                                           
--------------------------------------------------------------------------------------------------------------------------------
 Gather  (cost=1000.00..27236.53 rows=2541 width=79) (actual time=6.583..272.790 rows=2438 loops=1)
   Workers Planned: 2
   Workers Launched: 2
   ->  Parallel Seq Scan on submissions  (cost=0.00..25982.43 rows=1059 width=79) (actual time=2.572..252.628 rows=813 loops=3)
         Filter: ((user_id)::text = ANY ('{kimiyuki,kenkoooo}'::text[]))
         Rows Removed by Filter: 437021
 Planning time: 0.486 ms
 Execution time: 273.057 ms
(8 rows)

速度 張った後:

atcoder_problems=> CREATE INDEX ON submissions (user_id);
CREATE INDEX
atcoder_problems=> EXPLAIN ANALYZE SELECT * FROM submissions WHERE user_id IN ('kimiyuki', 'kenkoooo');
                                                              QUERY PLAN                                                               
---------------------------------------------------------------------------------------------------------------------------------------
 Bitmap Heap Scan on submissions  (cost=60.56..7102.58 rows=2541 width=79) (actual time=0.695..3.169 rows=2438 loops=1)
   Recheck Cond: ((user_id)::text = ANY ('{kimiyuki,kenkoooo}'::text[]))
   Heap Blocks: exact=1071
   ->  Bitmap Index Scan on submissions_user_id_idx  (cost=0.00..59.92 rows=2541 width=0) (actual time=0.460..0.460 rows=2438 loops=1)
         Index Cond: ((user_id)::text = ANY ('{kimiyuki,kenkoooo}'::text[]))
 Planning time: 0.180 ms
 Execution time: 3.475 ms
(7 rows)

btreeで張るかhashで張るかは両方試してどちらも変わらなかったのでとりあえずbtreeにしてます。

@codecov-io

This comment has been minimized.

Copy link

codecov-io commented Dec 22, 2018

Codecov Report

Merging #78 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master      #78   +/-   ##
=======================================
  Coverage   77.71%   77.71%           
=======================================
  Files          60       60           
  Lines        1315     1315           
  Branches       96       96           
=======================================
  Hits         1022     1022           
  Misses        288      288           
  Partials        5        5

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 45ab2b7...b644cab. Read the comment docs.

@kmyk

This comment has been minimized.

Copy link
Contributor

kmyk commented Dec 22, 2018

あっ CIが両方通る様になってる えらい

@kenkoooo kenkoooo merged commit cc62f31 into kenkoooo:master Dec 22, 2018

3 checks passed

codecov/patch Coverage not affected when comparing 45ab2b7...b644cab
Details
codecov/project 77.71% remains the same compared to 45ab2b7
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@kenkoooo

This comment has been minimized.

Copy link
Owner

kenkoooo commented Dec 22, 2018

ありがとうございます。本番DBにも反映しました。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment