-
Notifications
You must be signed in to change notification settings - Fork 135
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Speed issues? #53
Comments
What is your GPU usage? |
Gpu like 0-1% Going from memory when this issue appeared I believe it started once I made the changes to vectorize state. So I'm taking my CSV > Table.matrix then reshaping it. I've tested if it perhaps was due to reshaping taking significant time by using a constant state variable for this but it doesn't seem to have an effect. Could something be broken in my vectorize state that's causing this? Also - do you have discord or anything? |
Did you manage to train the connect four agent on your hardware and how much time did it take?
It seems possible but unlikely. Are you sure the problem is not simply that your states are too big for your hardware?
Unfortunately not. I am currently lacking the time to be present on discussion platforms such as Slack or Discord. |
Not fully, progress bar was filling quite quickly. I confirmed good gpu usage, unsure on cpu.
Hm I'll dig into this more...
Ok I'll try to take a look at this.
Makes sense as to how productive you are :) |
This means that the network is outputting a distribution that is not a valid probability distribution. Maybe should should add some print statements to see what's going on. Another source of concern here is the 98.4% redundancy figure, which may indicate you are not doing enough exploration. |
I fixed it - Would rigging this to multiple machines increase benchmark speed? |
Currently looking quite grim in terms of how many instances I'd need to generate this. |
Can you give me the numbers you are getting with Using multiple machines would increase speed, although right now only the self-play data generation is distributed, meaning that you would start hitting diminishing returns after about a dozen machines as training the network becomes the new bottleneck. (This will be improved in future releases but I am thinking about the best way to do it without adding too much complexity.) If your problem is too complex for the hardware you have, you may want to reformulate it so that you end up with smaller states, or use a smaller neural network architecture (or even use faster ML models such as gradient boosted trees). |
If I could figure out a way to integrate an LSTM into this it would 'cut the data by' 1500x, but even if I had that wouldn't I still need to store that state? I need to research alphazero further and get a deep enough understanding to understand how to solve this. But yeah in your expert opinion is there any solution to solve this 'refeeding' of data problem? |
Apparently Open.Ai's Alphazero stores previous states for it's calculations, does Alphazero.jl do the same in any capacity? |
I am not sure what you mean by this. Previous states are typically stored in the MCTS tree but there is no need to send previous states to the network (almost by definition of a state). More generally, I am wondering if you're not trying to apply AlphaZero on a problem where it does not really apply. Also, here is an advice. Despite many people's efforts to make AlphaZero more accessible (including my own efforts with this library), AlphaZero is not an algortihm you should be expecting to use on your problems as a black box. Even for using it on simple board games, you will need some deep knowledge about that algorithm so that you can tune the hyperparameters properly without spending too much compute resources. For more unusual applications, you should be ready to modify the implementation itself so as to generalize some components or integrate domain knowledge to improve sample efficiency. So my suggestion is to learn more about AlphaZero and make sure that AlphaZero is the most appropriate algorithm for solving your problem. Here, it may be useful to write a post on Julia's Discourse or an ML forum where you explain your problem clearly and where people can chime in and give suggestions on what approaches make sense. |
Think I figured out how to simplify it. It was quite obvious but somehow escaped me. Once again sorry to be a pain but I'm struggling with this bug. I'm struggling to figure out the cause of this. I've attempted to have debug prints to narrow down the cause. I can't tell where it's coming from. Perhaps something with vectorize states? |
Seemed to figure it out. It doesn't appear to like my 'clever' way to simplify things. |
So yeah is there anyway to continue a game whist forcing a single move? |
Final Update: I attempted to add an extra possible move that can always be made but it doesn't seem to fix the problem? I have no idea why this works but this recent logic I added causes it to break. Something regarding the actions and the amask |
I think it's something busted with state maybe? Like perhaps some sort of desync? Could this be do to set_state! and the fact I'm not setting all my variables from this state in set_state? |
I'm currently running into major speed issues.
![image](https://user-images.githubusercontent.com/78921582/127699325-168dd77e-69d8-4ce5-b5f0-07b21484e715.png)
The progress bar doesn't even appear.
Prints from my game seem to show everything working as intended.
My cpu usage is also quite low, around 8%.
Any tips for tracking down the cause of this?
The text was updated successfully, but these errors were encountered: