Skip to content
This repository has been archived by the owner on Dec 11, 2020. It is now read-only.

experimental branch tutorial_distri infinite loop #109

Closed
alatyshe opened this issue Oct 24, 2018 · 2 comments
Closed

experimental branch tutorial_distri infinite loop #109

alatyshe opened this issue Oct 24, 2018 · 2 comments

Comments

@alatyshe
Copy link

alatyshe commented Oct 24, 2018

Hi, I just wanted to find out. Why after starting server/client and transferring parameters --num_episode 4 to the server, it's simply ignored and both(server, client) just endlessly send batches to each other(as I understand, client sends and server only accepts).
And I notice that server gets stuck in GCWrapper method called run() on this line:
smem = self.GC.wait ()
2018-10-24 13 31 10
2018-10-24 13 31 30

Also I would to know why server sends message to client every 5-10 seconds:
TrainCtrl client: ReplyMsg [local-2341-rl-test-2-70c-4c32-ccee-2ef4]: {"state": {"content": 27}}
Where this is in the code?
Of course I understand that OnReply() method in TrainCtrl is being called, but it essentially stops the client and informs him that it's necessary to save everything in json and send it to the server right now.
I want the client just periodically send information about the passed stage (the client will call the nn itself), and the server would simply receive information and don't put on pause the client’s operations every 5-10 seconds.

@alatyshe alatyshe changed the title experimental branch experimental branch tutorial_distri infinite loop Oct 24, 2018
@yuandong-tian
Copy link
Contributor

Thanks for your interest! @alatyshe
The server will send the client message from time to time to make sure the client doesn't die. The server itself has a 20-minute (if I remember correctly) timer. If the client didn't respond then it is regarded as dead. If you open 2000 clients this is likely to happen.

@alatyshe
Copy link
Author

alatyshe commented Oct 25, 2018

Ok, I understand, but where can I change this timer, or is it even better to change the messages sent from the server to the client and vice versa?
I mean which class or method is response for that.
ps idk how to debug such a big project with dynamic libraries on remote server.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants