Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Faster then realtime training #626

Closed
vasaka opened this issue Apr 14, 2018 · 5 comments
Closed

Faster then realtime training #626

vasaka opened this issue Apr 14, 2018 · 5 comments
Labels

Comments

@vasaka
Copy link

@vasaka vasaka commented Apr 14, 2018

I have a questions about training of physics based environment. If it is training faster than realtime game how does it scales back to normal? It would seem that physics is dependent on system time in it's simulation not on step number and running on a different speed will completely change environment behavior from agent point of view.

Also what happens to calls dependent on Time.deltaTime when you run int training mode?

@vincentpierre

This comment has been minimized.

Copy link
Collaborator

@vincentpierre vincentpierre commented Apr 14, 2018

In order to run the simulation faster, we modify Time.TimeScale in unity. This can be used to run the simulation up to 100 times faster. You can see this int he academy inspector under Training Configuration. We try to synchronize the Updates with the Fixed Updates, but we recommend putting all logic into FixedUpdate rather than Update (and use Time.fixedDeltaTime instead of Time.deltaTime), that is because the agent's behavior runs in Fixed update. In all of the examples we provide, we can train at max speed (x100) and obtain similar behavior at x1 speed.
Does that help ?

@superjayman

This comment has been minimized.

Copy link

@superjayman superjayman commented Apr 14, 2018

vincent it seems it's not as simple as that! There is a strong correlation to processing power of the computer, at increased time scales joints in Unity behave differently( unstable). The ball demo is too simple, to notice this behavior, at 100x most things DO NOT train properly. It actually messes up the rewards when you log them. Why not leave it always at 100x for all simulations if it does not mess up anything?

Also, how does the Target Frame Rate play into all this?

@vincentpierre

This comment has been minimized.

Copy link
Collaborator

@vincentpierre vincentpierre commented Apr 14, 2018

I was not aware of this limitation, I train all my environments including crawler at 100 speed. my understanding was that the physics update always consider Time.fixedDeltaTime to be fixed and compute changes of velocity/position using this fixed delta time. If the computation takes longer, my understanding was that the game would slow down not that the physics would misbehave. Maybe there are physics computations that makes approximations when the resources are limited leading to instabilities.
From what I know the target frame rate corresponds to the number of screen updates per second, it should not affect the performance of the simulation if the logic is in fixedUpdate.

@vasaka

This comment has been minimized.

Copy link
Author

@vasaka vasaka commented Apr 14, 2018

Thanks, that helps

@xiaomaogy

This comment has been minimized.

Copy link
Collaborator

@xiaomaogy xiaomaogy commented May 17, 2018

Hi @vasaka @superjayman, we are closing this issue due to inactivity, feel free to open it if you guys want to discuss more on this issue.

@xiaomaogy xiaomaogy closed this May 17, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.