-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Convert Invoke Method to support async fully #31
Comments
All blocking IO to the Cache Repositories has now been moved out to async methods. Await not generally used and instead the tasks are joined at appropriate points in the flow (having the effect of increasing pressure on the thread pool but improving response times). In anticipation of the Redis project the invoke flow now writes out the latest event alongside distinct key value pairs, which increases storage but reduces the need for a very expensive distinct lookup in background Abstraction Rule value calculations (such a lookup is not even possible in the Redis model). All calls to the PostgresSQL database for caching now implement async methods only. The project should be close to being in a state where an interface can be extracted for all of the Cache repositories, and Redis equivalent to be instantiated as part of a builder process in the Invoke process. Background ThreadStart methods have been removed almost entirely and replaced with Task.Run as given the level of async decoration in the code it was unavoidable, although, should push more IO intensive tasks to the thread pool and overall ensure consistent performance. Removed legacy thread forking of Abstraction Rule processing from Invoke and the Environment Variables. Included a diagram in documentation that describes the flow and removed reference to the old thread forking method. Updated test to allow for parameter control of the number of allowed connections for HTTP to test load. Migration created for CachePayloadLatest table with required indexes. During payload lookup the currently in process event is explicitly excluded from the select given that insertion of in process transaction is one of the first tasks, and the availability of it from the database is inconsistent. The event is added to the transactions otherwise returned. |
See commit 5e62f10. |
The Redis project calls for reads to be performed against the Redis cache in parallel to the PostgreSQL Cache. More generally the async methods provide for a much better experience where the thread would otherwise be waiting on IO. The async methods have been proven to perform better than blocking methods by quite a margin and the total linear processing of the invoke method, putting aside the ForkAstractionKeys setting which is badly implemented.
Upgrade all PostgreSQL cache calls to fully support async methods.
Deprecate ForkAstractionKeys and make this the only processing method using task completion joining on wait complete.
The text was updated successfully, but these errors were encountered: