Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MetaSchedule][BugFix] Fix skipped tests #10885

Conversation

junrushao
Copy link
Member

@junrushao junrushao commented Apr 2, 2022

After #10366 is merged, all the search configurations (EvolutionarySearchConfig, ReplayFuncConfig, ReplayTraceConfig) are updated with a new field max_trials_global to be consistent with AutoScheduler's gradient-based task scheduler, which allows users to control the total number of trials of a model, while the scheduler distribute them unevenly according to task importance. This PR fixes the test cases that are not updated with this change, which are left undetected previously because they are not run in the CI.

For example, when tuning BERT-base, we could set max_trials_global to 20000, and the task scheduler will distribute trials to each task by preferring tasks with more potential performance growth:

 ID |                                                              Name |      FLOP | Weight | Speed (GFLOPS) | Latency (us) | Weighted Latency (us) | Trials | Terminated
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  0 |                                                        fused_take |         1 |      1 |         0.0005 |       2.1504 |                2.1504 |      5 |          Y
  1 |                                      fused_nn_dense_add_fast_tanh |   1204224 |      1 |       139.7438 |       8.6174 |                8.6174 |    192 |          Y
  2 |                       fused_reshape_add_reshape_transpose_reshape |     49152 |     12 |        19.5901 |       2.5090 |               30.1083 |      5 |          Y
  3 |                                                    fused_variance |    147520 |     25 |        68.8360 |       2.1431 |               53.5766 |     45 |          Y
  4 |                                                        fused_mean |     49216 |     25 |        22.8079 |       2.1578 |               53.9462 |     45 |          Y
  5 |                                               fused_cast_take_add |     49152 |      1 |        21.4777 |       2.2885 |                2.2885 |      5 |          Y
  6 |                     fused_reshape_add_reshape_transpose_reshape_1 |     49152 |     24 |        22.7076 |       2.1646 |               51.9496 |      5 |          Y
  7 |                                          fused_reshape_divide_add |     98304 |     12 |        45.9483 |       2.1394 |               25.6734 |      5 |          Y
  8 |                                             fused_nn_fast_softmax |   4374528 |     12 |      1350.8093 |       3.2384 |               38.8614 |    125 |          Y
  9 |                                                     fused_reshape |         1 |     12 |         0.0005 |       2.1827 |               26.1925 |      5 |          Y
 10 |                                             fused_nn_batch_matmul |   6291456 |     24 |      1427.9791 |       4.4058 |              105.7403 |    960 |          Y
 11 |                                   fused_reshape_transpose_reshape |         1 |     12 |         0.0005 |       2.1435 |               25.7225 |      5 |          Y
 12 |                                                    fused_nn_dense |  75497472 |     48 |      3810.7617 |      19.8116 |              950.9591 |   7424 |
 13 |                                                   fused_reshape_1 |         1 |     24 |         0.0005 |       2.1412 |               51.3892 |      5 |          Y
 14 |                                                  fused_nn_dense_1 | 301989888 |     12 |      6289.6575 |      48.0137 |              576.1647 |   4544 |
 15 | fused_reshape_add_multiply_fast_erf_multiply_add_multiply_reshape |  15532032 |     12 |      4535.0678 |       3.4249 |               41.0985 |      5 |
 16 |                                                  fused_nn_dense_2 | 301989888 |     12 |      4206.6203 |      71.7892 |              861.4703 |   6656 |
 17 |                                             fused_reshape_add_add |     98304 |     24 |        39.9511 |       2.4606 |               59.0547 |      5 |          Y
 18 |                       fused_subtract_add_sqrt_divide_multiply_add |    196672 |     25 |        72.8936 |       2.6981 |               67.4517 |      5 |          Y
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total trials: 20046
Total latency (us): 3032.42

CC @masahi

Copy link
Member

@masahi masahi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick fix!

@kparzysz-quic kparzysz-quic merged commit 6f0017a into apache:main Apr 2, 2022
pfk-beta pushed a commit to pfk-beta/tvm that referenced this pull request Apr 11, 2022
mehrdadh pushed a commit to mehrdadh/tvm that referenced this pull request Apr 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants