-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Runtime selection of fp16/fp32 #1649
Changes from 2 commits
035fe32
e9d1e7a
0122e42
c2ab6a9
23d2feb
7735fd0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -345,26 +345,42 @@ void Network::initialize(int playouts, const std::string & weightsfile) { | |
|
||
std::vector<ForwardPipe*> to_init; | ||
|
||
bool use_selfcheck = true; | ||
#ifdef USE_OPENCL | ||
if (cfg_cpu_only) { | ||
myprintf("Initializing CPU-only evaluation.\n"); | ||
m_forward = std::make_unique<CPUPipe>(); | ||
use_selfcheck = false; | ||
} else { | ||
myprintf("Initializing OpenCL.\n"); | ||
m_forward = std::make_unique<OpenCLScheduler>(); | ||
} | ||
#ifdef USE_HALF | ||
if (cfg_use_half) { | ||
myprintf("Initializing OpenCL (half precision).\n"); | ||
m_forward = std::make_unique<OpenCLScheduler<half_float::half>>(); | ||
use_selfcheck = false; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How badly would the self-check need to be relaxed for HALF to pass? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The reason I'm asking is that the comment for USE_HALF says "please test before enabling it". And the user is going to wonder: test what? 😁 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I did a bit experimenting and ran the whole thing overnight (with self-checking everything) and I recall that eventually failed even with 100% margins. Most of the cases the error was less than 1% so it was okay. I even moved the self-check to the final results (from the output of the forward() call) and it still did fail with 100% margins. It didn't seem to yield something too problematic (e.g., policy net moving a probability from 0.05 to 0.1) which isn't something that the tree search cannot fix, though. I guess if we still want the USE_HALF self-check we have to do something like 'N cases with average error of XXX%' but I am not sure what is the right value/way to do this. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The self-check has a catch that "rounds" all small values to zero (because the relative error gets very big on a very small value). Maybe it's just a matter of slightly pulling that up. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think Leela Zero Chess has a probabilistic self-check, in that it tolerates the occasional failure. Not sure how that combines with us already doing the self check once every xxx nodes though. At some point you're not going to catch buggy drivers either any more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Technically what we should test... is the strength of the engine (which one yields better win rate - speed vs. NN accuracy) and that can differ quite a bit depending on which GPU the user uses. The last time I tested, I figured out that high-bandwidth GPUs (e.g., Tesla P100 from Google Cloud) doesn't yield much additional performance hence we were sacrificing accuracy for nothing. In those cases --use-half would be worthless. Maybe all I can say for now is to delete the comment? :) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. One option would be to use KL divergence or some other measure to calculate the error in the self check. KL divergence doesn't care if the low probabilities are little off unlike the current method. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Okay, what I will do is:
|
||
} else { | ||
myprintf("Initializing OpenCL (single precision).\n"); | ||
m_forward = std::make_unique<OpenCLScheduler<float>>(); | ||
} | ||
#else | ||
myprintf("Initializing OpenCL (single precision).\n"); | ||
m_forward = std::make_unique<OpenCLScheduler<float>>(); | ||
#endif | ||
} | ||
|
||
#else //!USE_OPENCL | ||
myprintf("Initializing CPU-only evaluation.\n"); | ||
m_forward = std::make_unique<CPUPipe>(); | ||
use_selfcheck = false; | ||
#endif | ||
|
||
to_init.emplace_back(m_forward.get()); | ||
|
||
#ifdef USE_OPENCL_SELFCHECK | ||
if (!cfg_cpu_only) { | ||
if (use_selfcheck) { | ||
m_forward_cpu = std::make_unique<CPUPipe>(); | ||
to_init.emplace_back(m_forward_cpu.get()); | ||
} | ||
#else | ||
(void)use_selfcheck; | ||
#endif | ||
|
||
for (const auto& p : to_init) { | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Typo.