Avoid duplicate transfer #41
Conversation
|
Rishi, how this code will behave if agent will need to compare 1k-10k records? My understanding you want to compare dataset request with another agent. A dataset may contains 10k files. That's why I asked. Can you add unit/integration test to verify that both comparison function and request one will sustain a request with 10k records? |
|
@vkuznet First I am fetching all the files of requested data set from the source DB. After that, I am fetching the files of the same dataset from the destination agent. Then, I am storing destination's file name as a key and hash as its value in a separate map. This will help us to reduce the time complexity. The overall time complexity will be the O(N * m) where m is the search complexity of map. In many cases, it will be constant. Very soon I will add the unit tests. Can you suggest some better approach for this? |
|
@vkuznet I just added the test. Also, I did the benchmark of it. It is working fine.
|
|
ok, good, please fix %s in printouts of the test.
…On 0, Rishi ***@***.***> wrote:
@vkuznet I just added the test. Also, I did the benchmark of it. It is working fine.
Checkout screen shot.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#41 (comment)
|
|
Already, fixed that. Check out this commit. |
test/compare_test.go
Outdated
| ) | ||
|
|
||
| // Compare 1k files of source with the same 1k files of destination | ||
| func TestCompareThousand(t *testing.T) { |
There was a problem hiding this comment.
I would suggest either to delete this test since you have identical TenThousand one or make test to accept optional parameter for number of catalog entries, e.g. 1k or 10k or whatever. So far TestCompareThousand and TestCompareTenThousand are identical except one parameter in their body.
No description provided.