You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi,
We have structure ( Trade ) which has ID fields unique always.
We noticed if use Hash or Navigable index on ID field to push it into CqEngine
collection the size of heap taken is doubled up, which is baffling in a way.
SO basically adding a index where values are going to Unique is not working out
well for us. DO you have any recommendation on which index to be used and how
we can reduce the memory footprint here ?
The more indexes the heap size is getting lot worst, specially if the values
are unique and spread is high.
Any recommendations are highly appreciated.
Thanks,
Sandeep
Original issue reported on code.google.com by sandeepd...@gmail.com on 15 Oct 2013 at 6:58
The text was updated successfully, but these errors were encountered:
Hi Sandeep, this belongs more in the forum than as an issue.
Have you tried IndexQuantization? This is a good way to reduce memory usage.
Also are your ID values sparse? If so you could write your own quantizer to
better handle sparse values, like this one:
/**
* Converts sparse double values, to a fixed number of buckets.
* <p/>
* This quantizer should only be used with equality-based indexes, such as {@code HashIndex}.
*/
public class SparseDoubleQuantizer implements Quantizer<Double> {
private final int numBuckets;
public SparseDoubleQuantizer(int numBuckets) {
this.numBuckets = numBuckets;
}
@Override
public Double getQuantizedValue(Double attributeValue) {
return (double) attributeValue.longValue() % numBuckets;
}
}
The quantizer above will restrict the size of an equality-based index (e.g.
HashIndex) to a fixed number of buckets, independent of the number of unique
values of your IDs. But it cannot be used with a NavigableIndex.
This is not the best place to answer questions. If you like post your question
in the forum http://groups.google.com/group/cqengine-discuss and we can
continue discussion there. thanks!
Original comment by ni...@npgall.com on 15 Oct 2013 at 12:10
Original issue reported on code.google.com by
sandeepd...@gmail.com
on 15 Oct 2013 at 6:58The text was updated successfully, but these errors were encountered: