[AutoScheduler] Improve tuning with random cost model #6835
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@jcf94 please take a look at the failed test. It is because all measured schedules are invalid, and the root cause is that this PR changes the GA iteration to 3 when random cost model is used. It means this test is potentially flaky. |
The problem is that after this PR, that unit test will only try very few candidates, and it will become flaky if all candidates are invalid (e.g., PTX error). |
comaniac
force-pushed
the
ansor_fix_simple
branch
from
7a95727
to
f2666f7
Compare
|
@merrymercy @jcf94 I increased the measure trial from 2 to 10 to reduce the flaky possibility (please see the latest commit). |
merrymercy
approved these changes
Nov 11, 2020
merrymercy
approved these changes
Nov 11, 2020
trevor-m
pushed a commit
to trevor-m/tvm
that referenced
this pull request
Dec 2, 2020
* fix * more fix * fix * revert * format * Update sketch_policy.cc * increase measure trial to avoid flaky
trevor-m
pushed a commit
to trevor-m/tvm
that referenced
this pull request
Dec 4, 2020
* fix * more fix * fix * revert * format * Update sketch_policy.cc * increase measure trial to avoid flaky
trevor-m
pushed a commit
to neo-ai/tvm
that referenced
this pull request
Dec 4, 2020
* fix * more fix * fix * revert * format * Update sketch_policy.cc * increase measure trial to avoid flaky
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When tuning a task with all invective operators on GPU, Ansor inlines all operators for better performance. However, this makes the expression complex and causes high overhead of lowering the TE schedule, which is required for cost model feature extraction. Since the tuning space of these tasks is relatively small, it is sufficient to use random cost model so that we can avoid the lowering overhead.
However, the flow of InitPopulation -> RandomStates only gave me one state. After diving into details, I found that all states generated by initial populations are the same in terms of state.ToStr(). The reasons I can think of are either 2,048 is insufficient to produce two different states, or ToStr() does not reflect different states. Here is an example output of initial population sampling with de-duplication (throw away the states if it is already in the out_states):
As can be seen, even 50K samples cannot even produce the second state. I added a set to check the number of unique states before and after infer bound, and here is the results:
This log implies two points:
We should call ToStr() after infer bound to make sure we can differentiate states. As we can see from the log, 2,048 candidates have the same ToStr() before infer bound, but we got 7 different ToStr() after infer bound.
Systematically mutate tile sizes is way more efficient than random sampling. As can be seen from the log, we can get 7 different states in only 2,048 mutated states, but we only get 1 state in 43,007 random states.
Accordingly, this PR runs evolutionary search even we are using the random cost model and here is the new log (initial poulation is set to 1 and retry is set to 2):
cc @merrymercy @jcf94