Mapper and TableMapper performance is poor with large output records #1620

Closed
davidnich opened this Issue Jan 31, 2017 · 0 comments

Comments

Projects
None yet
2 participants
@davidnich
Contributor

davidnich commented Jan 31, 2017

When there are lots of identical mappings and/or constant values, each row iterated is processed one field at a time, which is inefficient and slow

Furthermore the bulk insert / upsert capabilities of TableMapper are much slower due to the fact that each row and each field must be processed individually.

see: https://docs.google.com/spreadsheets/d/1pTsV3mXZ6M0EUlszswMUdEdWEtiaEgxrwXmLQ7VM65A/edit#gid=0

@davidnich davidnich added this to the 0.8.13 milestone Jan 31, 2017

@davidnich davidnich self-assigned this Jan 31, 2017

@davidnich davidnich added feature and removed bug labels Jan 31, 2017

davidnich added a commit that referenced this issue Jan 31, 2017

davidnich added a commit that referenced this issue Jan 31, 2017

@davidnich davidnich modified the milestones: 0.8.12.6, 0.8.13 Jan 31, 2017

omusil24 added a commit that referenced this issue Feb 1, 2017

@omusil24 omusil24 added the fixed label Feb 1, 2017

@omusil24 omusil24 closed this Feb 1, 2017

davidnich added a commit that referenced this issue Mar 3, 2017

refs #1736 fixed a bug in processing "list mode" data in the Mapper c…
…lass such as submitted from InboundTableMapper::queueData(hash of lists); this bug was introduced in Qore 0.8.12.6 with the fix for refs #1620
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment