make.index.unique() should add a small eps to duplicate index values. This often works, but can fail when there are consecutive observations with the same timestamp, and first observation after the block of duplicate timestamps is less than the cumulative eps.
The functional change in this commit is adding eps when this is
newindex_real[i] <= newindex_real[i-1]
instead of when this is true:
index_real[i-1] == index_real[i]
This ensures the new index is always in increasing order, even
observation following a contiguous block of duplicate timestamps
less than the sum of the epsilons.
The memcpy ensures 'newindex' is equal to the current 'index'
the loop, which also means we longer need 'index_real'.
Main consideration here is if xts would be the class of choice for ultra low latency analysis in finance or other disciplines. If that is the case, as you have said earlier, support for more precise time indexing would be the solution. As I don't know the details of that solution, will leave it to you to weight implementation costs and potential issues with backwards compatibility.
I would like nanosecond resolution in xts, but that will take a bit of work. This problem could theoretically exist even with higher resolution index timestamps, but it would be less probable.
The current solution in this branch works around the issue by always checking and ensuring that the value of newindex[i] is always greater than both index[i-1]andnewindex[i-1]. The downside to this solution is that non-duplicate index values may change. I plan to add a warning whenever that happens before merging this branch and closing the issue. I can't think of a better general solution, but I'm open to suggestions.
I would like nanosecond resolution in xts, but that will take a bit of work.
I showed a possible solution to this problem (#190 (comment)) and yes, it is a lot of work, but in my opinion it is worth taking up this effort. The key is to not break xts R API for external packages.
If you decide on the proposed solution, then of course I will be very committed to help as much as possible (especially in the case of C code).
This can happen if the cumulative epsilon for a set of duplicate index
values is larger than the first unique index value that follows. We
will overwrite that non-duplicate index value with the prior index
value + eps when this happens, and warn the user.