Request for help running with Win10 #173
Comments
|
how many playouts per second are you getting on average with the 2400g w/vega 11? |
|
LizGoban reports v/s which I presume is visits/second. At move 99 of a game I am getting 110 v/s to 170+ v/s, mostly staying around 145+ v/s. I believe playouts/sec would differ? |
|
That's with the "KataGo" engine that is packaged with the current LizGoban. With Leela Zero in the same LizGoban version it is around 40 v/s. |
|
Your Vega 11 GPU is being utilized correctly by both katago and leela zero |
|
When you install baduk megapack, sabaki should work well. Under "manage engines" see how they format the commands and copy that format when adding your own engines. Using the newest 20b katago, I would say 350 playouts/move would give you optimal analysis vs time ratio for your PC, so under 15 seconds per move with a bot strong enough to beat some weak pros |
That's really good to hear. Many thanks for responding and checking it out for me! That answers (1). I hope someone will be able to help with (1a) below and with (2) to (5) in the original post. (1a) Can you give me an idea of how good this performance is, compared to say medium to high performance GPUs? A few examples would be good. I might add a second GPU if it is worthwhile (and not exorbitantly expensive).
What (if any) is the relation between playouts/move and visits/move? Re Sabaki:
In my Sabaki v0.50.0 nothing displays in the "Manage Engines" boxes/fields. I've tried emulating commands and format that I've seen on YouTube in several different ways/variations but without success. Anyway I don't know how well the analyisis and various display windows in Sabaki compare with Lizzie and LizGoban. I like Lizzie but it hasn't been updated for 6 months. |
|
LizGoban running the packaged "KataGo" engine seems to be the best on my setup. (I prefer the Lizzie interface, including the way it displays information.) |
|
A gtx 1060 would give you double the playouts. I forgot how playouts is related to nn batches and nn rows but if you set threads to one in the config file, playouts will equal nn rows or nn batches in the gtp output. Visits includes searches from previous moves that are reused again, for example if your opponent plays an expected next move in the sequence. That's all I know. The only thing I can say about the newer Sabaki is to try installing the baduk megapack with the older sabaki, katago will show up in the manage engines area. Maybe there is not that much of advantage in using the newer one. Make sure your net has bin somewhere in the extension |
|
@portkata Thanks and Good Health! |
|
@portkata - thanks for helping out here! @GopherAl - do you have any remaining questions at this point? For some of your questions, it's really up to you - like how many visits to use depends on how long you are willing to let analysis take - more visits will always be stronger and more accurate, but will take longer, so it's entirely up to your preference. Just experiment with it and see what settings you find you personally are most satisfied with. If you are failing to run a model, then possible reasons for failure could be - you aren't using the latest KataGo version (if it was pre-packaged from somewhere), or you renamed the model incorrectly (there is an older |
|
@lightvector Thank you for responding. I'm using a recent version of LizGoban which includes two versions of KataGo (KG). Both versions seem to perform okay on my system (albeit slower than desirable). I believe that the "KataGo" version (as opposed to the "KataGo for handicap games" version) uses the net g170e-b15c192-s1672170752-d466197061 which I read somewhere (now I can't find it) is suitable for relatively weak machines. (No doubt mine falls into this category). My system is a Ryzen APU running the latest Win10: 1. Can you confirm that my GPU is working correctly? See usage percentage below. It’s hard to believe that I can run this software only using 2% of my fairly low range GPU.
(1a) Can you give me an idea of how good this performance is, compared to say medium to high performance GPUs? A few examples would be good. I might add a second GPU if it is worthwhile (and not exorbitantly expensive). 2. (a) I don't understand what @portkata is saying in their above answer. Firstly, I only get visits/move reported. In item 3 below I am looking for guidance on determining optimal analysis vs time used. 3. I am trying to work out how many visits to each of the top move choices are required to validate them as being the best moves? (My observations are that KG settles on the #1 move quite early on. Then it appears to mainly stick to analysing that line to greater depth.
4. What GUI do you use/recommend? Lizzie has certain advantages over LizGoban but I failed (KG stopped working) when I tried to change the net to g170e-b15c192-s1672170752-d466197061. (I assume that this is the net named "KataGo" in the "Preset" menu in LizGoban. Whatever that net is, it seems to work the best of those I've tried, including Leela Zero nets in Lizzie and LizGoban.) 5. Please advise what paths, commands, arguments, etc, that I need to enter in Sabaki/Add Engine fields. I have KataGo v1.3.3 in C:\Program Files\KataGo (This is totally independent from the BadukMegapack folders; I only tried installing things from BadukMegapack after I had the problems described previously.) It would be helpful if you also suggest what parameters I could try varying once I get KG running in Sabaki. It would also be great if you could give more detail on the steps required to use different nets.
Presumably this is what I am talking about in items (2) & (3)? I know you have much more important things to do! So take as much time as you like to reply to give you time to reply as fully as you can be bothered to. I can plug along with my current setup while I wait. Thanks for your great work! It is fantastic to have such great go software available! |
|
1: At the top end, a 2080ti should give you several thousand playouts per second with enough search threads. Somewhat weaker GPUs will give you low thousands or hundreds of playouts per second. There's a whole spectrum of possible levels of performance, I had a laptop integrated graphics which got like 10 playouts per second or something. Don't ask me where your GPU falls, I don't know what all the possible GPUs are. :) If both Leela Zero and KataGo cause you to report only a few % of GPU usage, and you seem to be getting okay performance, then I think the monitor for GPU usage is buggy, so if that's the case then just ignore it. 2, 3: You will get more playouts per second with a stronger GPU. There's not much more to say, really. Most of your questions here don't make sense. Why? Because there is no such thing as the optimal visits to use. More visits will always improve the average strength and accuracy, with each visit contributing gradually less and less on average, but still helping. With 1 visit, it should be high amateur dan level, and as you get into 100s of visits, it should be reaching into pro levels, even though there will also be many mistakes, and as you go into 1000s of visits, further and further superhuman. Tens of thousands, millions etc will push further and further into making fewer mistakes and being more accurate. There is no magic "optimal" number like 7k ,or 10k, or 100k, where anything special happens, it's all a smooth continuum. There is not even any number where below that number each visit helps a lot, and above that number each visit suddenly helps way less. On average, it's all very gradual and smooth. So as I said before, it's entirely your personal preference about how much you are willing to trade off accuracy versus time. Your questions have no answer because what tradeoff you prefer here is your own preference, and only yourself you can experiment and learn what you prefer. 4: Use whatever GUI you like the best. Try them all and see what you prefer. Again, don't ask me for answers about things that should be your own personal preference, you know better than me what you like once you try out a thing. :P 5: How about you instead give detail about exactly what you did and what files you download and where you put them, and what you typed in to what box and what went wrong, etc.? It's more efficient that way and easier for the person answering, because the person answering doesn't have to explain things that you already did correctly, and doesn't have to waste time to explain things that might be irrelevant (because there can be multiple ways that work, and they can't predict in advance which way you tried, or which ways are relevant for your system). Post screenshots if necessary. |
|
By the way, regarding 2,3 - it is frequently the case that the best move will be decided on at some number of visits, and never again will KataGo change its mind. How to reconcile this with the smoothness that I'm telling you above? Well, for some moves it will be decided be at hundreds of visits, and for some it will be thousands, and for some it will be tens or hundreds of thousands, some will be millions, etc. And the further you go, it will be fewer and fewer moves, gradually less and less common that it changes its mind, but never zero - statistically across many moves, more visits will always have a chance to help. And you can't know in advance which of these moves will be decided at what number of visits. So it is sometimes possible that a move "appears" decided at 1000 visits, and then at 100k visits KataGo will change its mind, etc. So on average the strength scales smoothly with visits, and there is no such thing as any single "optimal" number, and neither can you be ever absolutely 100% sure in advance which moves will be which, in general. |
|
baduk megapack was updated today, it now comes with the 40b, 30b, 20b, 15b, 10b, and 6b katago bots all preinstalled in Sabaki. I would be shocked if that doesn't work for you. good luck! and thanks lightvector for the thorough replies! |
|
@lightvector Many thanks for your comprehensive explanations! It makes things much clearer for me. @portkata Thanks again to you. I'll try the new baduk megapack. @lightvector Please comment on the below points. As you can see, I have limited knowledge of all of this. I expect that your answers to the following questions will tell me most, if not all, of what I need to know.
1. My interpretation of the above LizGoban diagram is:
4. My understanding is that Leela Zero eventually will evaluate every empty point on the goban whereas you take a targeted approach and KataGo only evaluates a subset of the empty points. Is this correct or will KataGo eventually look at all points?
5. I am still concerned that my GPU is not being utilised. If true that would be most undesirable! How do I know whether or not I'm "getting okay performance"? My system runs for many hours to produce 4.5m visits. Whether this is okay or not, and whether it correlates with the GPU engine being utilised, I don't know. KataGo does quickly visit and rank a large number of points/moves (15 or so after a total of 50k visits). This could well be very acceptable, depending on your answer to (2). I interpret the katago.exe usage reported by Win10 Task Manager as follows. The fact that “GPU 0 – Copy” is displayed in the “GPU engine” column means (I hope) that KataGo is calculating using the GPU engine (although the 1.6% reported usage would be incorrect). Similarly, Radeon Software reports that my GPU usage is "N/A". I can think of two possibilities for this: Win10 Task Manager also reports 3.8% usage of the CPU by katago.exe. I would expect much higher CPU usage than that if my GPU is not being utilised. However I don't have any expertise. Can you throw any light on the above speculations? |
|
I have never used LizGoBan myself so I don't fully know how it selects its display, but... Does LizGoBan show winrate percentages or score leads by default? It possibly depends on how you configured it. In either case, whether a winrate or a score, the number is the result after the move, rather than the gain from the move. 2: No, that move would probably be a terrible blunder and KataGo with 1 visit would never play it. When I say KataGo with 1 visit is high dan amateur, I mean that if you let it analyze the whole board for only 1 visit total for the entire board on a given turn, the move it selects to put that 1 visit into will be on average good enough to be high dan amateur. Here in the screenshot it looks like you have about 402K visits total on the entire board. With 402k visits, it will consider placing some visits on less-likely moves, including moves that normally it would never play, just in case they are actually good, since it has so many to spare. But here it put one visit in that move, immediately disliked the result, and never considered that move again. If KataGo had only 1 visit for the entire board, that move would almost certainly not be its first choice, the only reason it even got 1 visit despite being a bad move was because there were so many visits available in total. 3: They are mostly the same. The only difference is when turn by turn you remember what you analyzed before. If I spend say 200 visits total on a turn and 100 of them are in move X, and then I play move X, and for 50 of those 100 visits they had explored the variation of the opponent to respond with move Y (you cannot see this in any GUI easily, but every visit explores a particular variation), and the opponent does indeed respond with move Y, then starting next turn I already have 50 visits spread across various moves on that board position before I have done any new thinking, already helping choose between different moves. If I then explore another 200 new visits, we would say that this turn, I have 250 visits total, but I have done 200 playouts. Playouts mean simply the "new" visits, without counting the exploration that could be saved and re-used from previous turns. 4: No, KataGo will eventually evaluate every move on the board unboundedly many times. It may take a long time though. If you would like to make it search more widely, you can add 5: I'm going to ignore everything you wrote there because I don't know much about your computer and I don't have a magical way to reach across the internet to debug your computer for you. :) How would you tell if you get a "reasonable" performance and therefore that the reported usage is just buggy? You would find a variety of other users and ask them what performance they got with their GPUs and compare with what you get, and then you would go online and look at official benchmarks about the other person's GPU versus your GPU and see if approximately in line with such benchmarks, or if in fact your GPU is giving you 30x slower performance than you would expect based on this. |
|
1ii/2: Also, because it only got 1 visit, the "1.7" on that move is extremely unreliable. You should not trust it at all, unlike moves with many visits which have been analyzed deeply and have accurate numbers, the "1.7" is only the shallowest-possible first-instinct guess. It's a high-amateur-dan guess, but still a guess, high amateur-dan-level still has lots of mistakes. The only thing you can trust is that KataGo dislikes that move sufficiently that it doesn't think it's worth the time to improve the accuracy of that estimate. If you actually want an accurate estimate for that move, the only way is to physically play it on the board and then see what scores or winrates KataGo gives you after that. So even if it is bad we play this move anyway, now this is the new board position, spend thousands of visits analyzing the results after that, and now whoever is ahead by how much will tell you what the result of that move actually is. |
|
@lightvector Many thanks for that. I'll ruminate on what you have said. Keep up the excellent work! (Good Health should be the salutation these days.) So Good Health! |
|
Closing this issue since it seems everything has been handled. |
KataGo is great!!! I also commend @lightvector on the excellent documentation! I have run KG with Lizzie and LizGoban with the packaged KG versions of each. My system is a Ryzen APU running the latest Win10:
I have two main issues:
1. It appears to me that my GPU is not being utilised by the engine because on my system I would expect somewhere around 100% utilisation of the GPU. However with LizGoban running KataGo the GPU uses about 3% GPU. With Lizzie (Java Platform SE binary (32 bit)) the GPU uses about 0.4% GPU running KataGo and about 0.2% GPU running LeelaZero.
With Leela 0.11 Graphics Accelerated the GPU uses about 4% GPU. Am I correct in assuming that this means that the engine is using the GPU in this instance? (since this software is relatively old and, I assume, undemanding).
2. I don't know how to swap networks over. (I have had no success in doing this with KataGo. However I think I succeeded with Leela Zero in Lizzie. I replaced the packaged version of Leela Zero with the one in the current download by moving_lznetwork.gz_ into a new folder and replacing it with the version included with the current Leela Zero download. It works but seems slow and weak.)
In addition I would appreciate advice on the following:
3. How many total playouts do I need to have in order to get the optimum analysis of best moves?
4. How many visits/playouts should there be for each suggested move to get the optimum analysis of best moves?
5. Which GUI do you recommend that I use? (All things being equal, I prefer Lizzie to LizGoban.)
I use Sabaki extensively but I've had no success with getting any engine to run on it (not even Leela 0.11). Help with this will also be gratefully received!
I have separately also tried installing and running Lizzie and LizGoban using the BadukMegapack. The setup process worked well, including setup and tuning of OpenCL and setting parameters. However neither Lizzie or LizGoban would work from the versions in the folders set up by BadukMegapack. (The stand-alone versions that I installed earlier still work as before.)
My impression from my experimenting is that LZ runs slowly and suggests "silly" moves whereas KG only suggests "sensible" moves. It is likely that this is because LZ has a larger block and my machine is relatively low spec. I prefer KataGo's graphics and the details given. KG also seems to work better on my machine than does LZ.
I like the way Lizzie auto-saves the SGF and when running saves it's analysis so that it is able to utilise previous analysis to give it a head start when I try a different move and when I follow one of the suggested variations.
LizGoban's two versions of KataGo both seem to perform okay on my system (albeit slower than desirable). I believe that the "KataGo" version (as opposed to the "KataGo for handicap games" version) uses the net g170e-b15c192-s1672170752-d466197061 which I read somewhere (now I can't find it) is suitable for relatively weak machines. (No doubt mine falls into this category).
I would greatly appreciate responses to my issues and also on my various speculations. Thanks in advance for your time and help!
The text was updated successfully, but these errors were encountered: