InboundTableMapper::queueData() error with hashes of lists #1736

Closed
davidnich opened this Issue Mar 3, 2017 · 3 comments

Comments

Projects
None yet
2 participants
@davidnich
Contributor

davidnich commented Mar 3, 2017

mapper fields that do not have any input data (ex: a closure that does not process input data for example) cause the following exception to be raised: User exception: /Users/david/src/qore/git/qore/qlib/Mapper.qm:1186: MAPPER-FIELD-LIST-ERROR: datasource "oracle:omquser@el7": field "dte" value passed is "nothing" in list mode; expecting "list"

this bug was introduced in Qore 0.8.12.6 with the fix for #1620

@davidnich davidnich added this to the 0.8.12.8 milestone Mar 3, 2017

@davidnich davidnich self-assigned this Mar 3, 2017

davidnich added a commit that referenced this issue Mar 3, 2017

refs #1736 fixed a bug in processing "list mode" data in the Mapper c…
…lass such as submitted from InboundTableMapper::queueData(hash of lists); this bug was introduced in Qore 0.8.12.6 with the fix for refs #1620

davidnich added a commit that referenced this issue Mar 3, 2017

refs #1736 added additional fixes and tests for compatibility with us…
…ing optimized hashes of lists and constant values

davidnich added a commit that referenced this issue Mar 4, 2017

refs #1738 fixed HashListIterator to correctly iterate hashes with mi…
…xed lists and single values and to throw a runtime exception if lists of different sizes are encountered instead of silently ignoring the data

refs #1736 updated the TableMapper class to use the original, must more efficient implementation of iterating hash of list data with <hash>::contextIterator()

added tests + relnotes
@pavelkveton

This comment has been minimized.

Show comment
Hide comment
@pavelkveton

pavelkveton Mar 6, 2017

Contributor

some more info from David about when the problem manifests itself:

[11:58:05 AM] David Nichols: most of the "code" closures don't use any input value - meaning the first argument
[11:58:17 AM] David Nichols: meaning that the output field has a different name than the input field
[11:58:46 AM] David Nichols: however the bug will only be seen when you submit data in "hash of lists" format to InboundTableMapper::queueData()
[11:58:51 AM] David Nichols: so normally this happens with DB input
[11:59:14 AM] David Nichols: so in other words, when you have DB input and then pass that input to an InboundTableMapper object with queueData() - you will see the bug

about the impacted structures of the mapper:

[12:44:52 PM] David Nichols: no it doesn't affect constants
[12:45:04 PM] David Nichols: because they were separated out and not passed to Mapper::mapFieldIntern()
[12:45:17 PM] David Nichols: it affects any output field that has no input field
[12:45:26 PM] David Nichols: so I think closure might be the only one
Contributor

pavelkveton commented Mar 6, 2017

some more info from David about when the problem manifests itself:

[11:58:05 AM] David Nichols: most of the "code" closures don't use any input value - meaning the first argument
[11:58:17 AM] David Nichols: meaning that the output field has a different name than the input field
[11:58:46 AM] David Nichols: however the bug will only be seen when you submit data in "hash of lists" format to InboundTableMapper::queueData()
[11:58:51 AM] David Nichols: so normally this happens with DB input
[11:59:14 AM] David Nichols: so in other words, when you have DB input and then pass that input to an InboundTableMapper object with queueData() - you will see the bug

about the impacted structures of the mapper:

[12:44:52 PM] David Nichols: no it doesn't affect constants
[12:45:04 PM] David Nichols: because they were separated out and not passed to Mapper::mapFieldIntern()
[12:45:17 PM] David Nichols: it affects any output field that has no input field
[12:45:26 PM] David Nichols: so I think closure might be the only one
@pavelkveton

This comment has been minimized.

Show comment
Hide comment
@pavelkveton

pavelkveton Mar 6, 2017

Contributor

so if I understand it well, it affects all the mappers with a closure used for DB->DB mapping with Bulk operations.

Mind that M_DATE is also a closure.

Contributor

pavelkveton commented Mar 6, 2017

so if I understand it well, it affects all the mappers with a closure used for DB->DB mapping with Bulk operations.

Mind that M_DATE is also a closure.

@davidnich

This comment has been minimized.

Show comment
Hide comment
@davidnich

davidnich Mar 6, 2017

Contributor

@pavelkveton yes as long as bulk DML is used for both input and output

Contributor

davidnich commented Mar 6, 2017

@pavelkveton yes as long as bulk DML is used for both input and output

pavelkveton added a commit that referenced this issue Mar 6, 2017

Merge pull request #1737 from qorelanguage/bugfix/1736_Mapper_list_mo…
…de_fix

refs #1736 fixed a bug in processing "list mode" data in the Mapper c…

@davidnich davidnich added the fixed label Mar 7, 2017

@davidnich davidnich closed this Mar 7, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment