This repository has been archived by the owner on Jul 11, 2022. It is now read-only.
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[BZ 185375] lots of clean up, refactoring, and perf improvements
The TaskTracker class has been copied into the source tree from the service-metrics maven module. The reason for copying the class is beause the changes for this BZ are going into a patch release, and I want to minimize the number of modules that have to be touched for this. We no longer query the rdbms to load schedule ids. Instead we use the legacy thrift APIs via the hector library to scan for schedule ids. This way, we query for only those schedule ids that have data. Reads are now throttled using a RateLimiter. This is better than using a semaphore for a couple reasons. First, we do not have to worry about releasing permits, which had to be done in a couple different places in the code. Secondly, it is more expressive about the throughput. With a semaphore, it was hard to determine the read throughput, whereas with RateLimiter it is directly specified as querys per second. The migration code previously tried to abort processing if a write failed. This made the code overly complicated. Since we are now using batches, we only do a handful of writes per schedule id. We now just have the failure detection and handling code in the migration finished callback. This simplifies things a lot.
- Loading branch information