Skip to content
This repository has been archived by the owner on Jan 18, 2020. It is now read-only.

Update exporter force_distinct feature to consider the whole iterable #174

Closed
bruth opened this issue Feb 17, 2014 · 0 comments
Closed

Update exporter force_distinct feature to consider the whole iterable #174

bruth opened this issue Feb 17, 2014 · 0 comments
Assignees

Comments

@bruth
Copy link
Contributor

bruth commented Feb 17, 2014

Currently, the implementation of the force_distinct flag in the exporter class will only remove consecutive duplicate rows. Depending on the sorting method, this could still produce duplicate rows later in the result set which is undesirable.

One caveat to this is that result set will be kept in memory for duration of the export. In practice, this is rarely an issue since the size of iterables are relatively small, but for large exports this could be very slow and potentially cause OutOfMemory errors.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant