Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] AmgX adjustments for benchmarking #15

Open
wants to merge 17 commits into
base: master
Choose a base branch
from
Open

Conversation

artv3
Copy link
Collaborator

@artv3 artv3 commented Dec 15, 2020

This PR makes adjustments to python scripts to set apart amg setup time, no of iterations, and time-per-iteration.
Still a work in progress, but logging what I see.

    mfem   |    |  comp  |     |number of |  amg       | number of  |               | cg-iter dps
   device  |  p |  iler  |  np |unknowns  |  setup     | iterations | time-per-iter |   millions
-----------+----+--------+-----+---------+-------------+---------+---------------+-------------
      cuda |  1 |    xlc |   4 |     18  |    0.000468 |       1 |    0.000000 |   0.033255
      cuda |  1 |    xlc |   4 |     27  |    0.000438 |       2 |    0.001043 |   0.038509
      cuda |  1 |    xlc |   4 |     45  |    0.000471 |       3 |    0.000981 |   0.057660
      cuda |  1 |    xlc |   4 |     75  |    0.000491 |       7 |    0.000940 |   0.087167
      cuda |  1 |    xlc |   4 |    125  |    0.000497 |       8 |    0.000969 |   0.139263
      cuda |  1 |    xlc |   4 |    225  |    0.000605 |      11 |    0.000931 |   0.255691
      cuda |  1 |    xlc |   4 |    405  |    0.000924 |      11 |    0.000950 |   0.450530
      cuda |  1 |    xlc |   4 |    729  |    0.004724 |      11 |    0.001284 |   0.607944
      cuda |  1 |    xlc |   4 |   1377  |    0.005715 |      15 |    0.000986 |   1.453810
      cuda |  1 |    xlc |   4 |   2601  |    0.009194 |      16 |    0.001372 |   1.984690
      cuda |  1 |    xlc |   4 |   4913  |    0.015172 |      16 |    0.001390 |   3.707380
      cuda |  1 |    xlc |   4 |   9537  |    0.036640 |      22 |    0.001878 |   5.273700
      cuda |  1 |    xlc |   4 |  18513  |    0.039395 |      14 |    0.001856 |  10.590700
      cuda |  1 |    xlc |   4 |  35937  |    0.052646 |      14 |    0.002336 |  16.330200
      cuda |  1 |    xlc |   4 |  70785  |    0.097023 |      27 |    0.003166 |  23.070800
      cuda |  1 |    xlc |   4 | 139425  |    0.099476 |      14 |    0.002876 |  51.618500
      cuda |  1 |    xlc |   4 | 274625  |    0.140822 |      12 |    0.003380 |  87.708900
      cuda |  1 |    xlc |   4 | 545025  |    0.381351 |      31 |    0.004954 | 113.373000
      cuda |  1 |    xlc |   4 | 1081665  |    0.357588 |      14 |    0.005256 | 220.212000
      cuda |  1 |    xlc |   4 | 2146689  |    0.749115 |      11 |    0.008046 | 291.702000
      cuda |  1 |    xlc |   4 | 4276737  |    2.608220 |      37 |    0.019247 | 228.178000
      cuda |  1 |    xlc |   4 | 8520321  |    2.131120 |      15 |    0.021306 | 427.370000

@artv3
Copy link
Collaborator Author

artv3 commented Dec 17, 2020

When using AmgX as a preconditioner, I can distinguish the cost of applying AMGX as the preconditioner, along with the PCG iteration cost (which I believe includes applying preconditioner):

    mfem   |    |  comp  |     |number of |  amg   | cost of  | number of  | time per | cg-iter dps
   device  |  p |  iler  |  np |unknowns  |  setup |  precon  | iterations | cg step |   millions
-----------+----+--------+-----+---------+---------+----+---------+---------------+-------------
      cuda |  1 |    xlc |   4 |     18  |    0.000374 |  0.00024474 |       0 |         inf |   0.000000
      cuda |  1 |    xlc |   4 |     27  |    0.000381 |  0.00023962 |       1 |    0.001414 |   0.019094
      cuda |  1 |    xlc |   4 |     45  |    0.000403 |  0.00023654 |       3 |    0.000944 |   0.047687
      cuda |  1 |    xlc |   4 |     75  |    0.000413 |  0.00022106 |       6 |    0.000833 |   0.090035
      cuda |  1 |    xlc |   4 |    125  |    0.000410 |  0.00023347 |      11 |    0.001171 |   0.106763
      cuda |  1 |    xlc |   4 |    225  |    0.000509 |  0.00022026 |      16 |    0.000970 |   0.231880
      cuda |  1 |    xlc |   4 |    405  |    0.000726 |  0.00024685 |      15 |    0.000794 |   0.510359
      cuda |  1 |    xlc |   4 |    729  |    0.004634 |  0.00020685 |      15 |    0.000996 |   0.731794
      cuda |  1 |    xlc |   4 |   1377  |    0.005256 |  0.00025005 |      23 |    0.000915 |   1.504310
      cuda |  1 |    xlc |   4 |   2601  |    0.009244 |  0.00034202 |      23 |    0.000944 |   2.754890
      cuda |  1 |    xlc |   4 |   4913  |    0.014462 |  0.00047923 |      24 |    0.001153 |   4.260620
      cuda |  1 |    xlc |   4 |   9537  |    0.036590 |  0.00072102 |      32 |    0.001345 |   7.090490
      cuda |  1 |    xlc |   4 |  18513  |    0.038512 |  0.00070694 |      21 |    0.001435 |  12.903100
      cuda |  1 |    xlc |   4 |  35937  |    0.051612 |  0.00097382 |      21 |    0.002042 |  17.598400
      cuda |  1 |    xlc |   4 |  70785  |    0.097607 |  0.00129322 |      36 |    0.002544 |  27.826700
      cuda |  1 |    xlc |   4 | 139425  |    0.097561 |  0.00129434 |      20 |    0.003259 |  42.778100
      cuda |  1 |    xlc |   4 | 274625  |    0.132797 |  0.00143853 |      17 |    0.004896 |  56.086000
      cuda |  1 |    xlc |   4 | 545025  |    0.385704 |  0.00242182 |      39 |    0.008543 |  63.795000
      cuda |  1 |    xlc |   4 | 1081665  |    0.351429 |  0.00253306 |      19 |    0.014508 |  74.556000
      cuda |  1 |    xlc |   4 | 2146689  |    0.732958 |  0.00396192 |      15 |    0.028630 |  74.979900
      cuda |  1 |    xlc |   4 | 4276737  |    2.626320 |  0.00967741 |      50 |    0.057062 |  74.948400
      cuda |  1 |    xlc |   4 | 8520321  |    2.138930 |  0.01042890 |      20 |    0.109639 |  77.712200

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant