Execute functions up to 5 seconds faster #1308
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Updated INTERVALS of php-resque
In general, this PR speeds up functions, webhooks and tasks by up to 5 seconds. Also, it speeds up the overall speed of the appwrite.
Test Plan
I run a whole project on Appwrite using functions and real-time feature.
Before using this fork, the time between running
createExecuton
and receiving the update event using real-time took 4-5 seconds.Using this fork, I reduced the time to 1s.
Addational research info
I tested everything on a function that takes ~1s to execute (load libraries into NodeJS, connect to MySQL server, get some data, process them and console log result). If a request takes 1s, it means 0ms delay due to Appwrite.
Currently, Appwrite uses the default INTERVAL configuration which is 5 seconds:
https://github.com/resque/php-resque/blob/master/lib/Resque.php#L13
https://github.com/resque/php-resque/blob/master/lib/Resque/Worker.php#L160
By changing this number to 0.1s, the MAXIMUM possible delay was reduced from 5000ms to 100ms. This resulted in really fast function executions but also highly increased CPU usage:
I switched to INTERVAL=1 which fixed the high CPU and still increased the speed of every request by up to 4 seconds:
Then I wanted to see what happens when I set INTERVAL=1 on everything but set INTERVAL=0.1 on the functions container. This was the result:
Also, of course, MAXIMUM of 100ms delay between creating and starting execution.
Finally, I decreased INTERVAL on functions, webhook and tasks real-time to 0.1 which resulted in these stats:
Current PR sets INTERVAL=1 on everything except functions, tasks and webhooks. These have even faster INTERVAL=0.1
I am not fully sure what to think about these stats, so it will need some more benchmarking but these screenshots should provide solid ground.