Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(query): query to csv skip empty lines #536

Merged
merged 2 commits into from
Dec 7, 2022
Merged

Conversation

bednar
Copy link
Contributor

@bednar bednar commented Dec 6, 2022

Closes #533

Proposed Changes

The querying into CSV skip empty lines from InfluxDB response.

#datatype,string,long,dateTime:RFC3339,double,string
#group,false,false,false,false,true
#default,_result,,,,
,result,table,_time,_value,_field
,,0,2022-11-24T10:00:10Z,0.1,_1_current_(mA)
,,1,2022-11-24T10:00:10Z,4,_1_current_limit_(mA)
,,2,2022-11-24T10:00:10Z,1,_1_voltage_(V)
,,3,2022-11-24T10:00:10Z,1,_1_voltage_limit_(V)
,,4,2022-11-24T10:00:10Z,0,_2_current_(mA)
,,5,2022-11-24T10:00:10Z,0,_2_current_limit_(mA)
,,6,2022-11-24T10:00:10Z,0,_2_voltage_(V)
,,7,2022-11-24T10:00:10Z,0,_2_voltage_limit_(V)

#datatype,string,long,dateTime:RFC3339,string,string
#group,false,false,false,false,true
#default,_result,,,,
,result,table,_time,_value,_field
,,8,2022-11-24T10:00:10Z,K,type
,,9,2022-11-24T10:00:10Z,,type2

is parsed as:

['#group', 'false', 'false', 'false', 'false', 'true']
['#default', '_result', '', '', '', '']
['', 'result', 'table', '_time', '_value', '_field']
['', '', '0', '2022-11-24T10:00:10Z', '0.1', '_1_current_(mA)']
['', '', '1', '2022-11-24T10:00:10Z', '4', '_1_current_limit_(mA)']
['', '', '2', '2022-11-24T10:00:10Z', '1', '_1_voltage_(V)']
['', '', '3', '2022-11-24T10:00:10Z', '1', '_1_voltage_limit_(V)']
['', '', '4', '2022-11-24T10:00:10Z', '0', '_2_current_(mA)']
['', '', '5', '2022-11-24T10:00:10Z', '0', '_2_current_limit_(mA)']
['', '', '6', '2022-11-24T10:00:10Z', '0', '_2_voltage_(V)']
['', '', '7', '2022-11-24T10:00:10Z', '0', '_2_voltage_limit_(V)']
['#datatype', 'string', 'long', 'dateTime:RFC3339', 'string', 'string']
['#group', 'false', 'false', 'false', 'false', 'true']
['#default', '_result', '', '', '', '']
['', 'result', 'table', '_time', '_value', '_field']
['', '', '8', '2022-11-24T10:00:10Z', 'K', 'type']
['', '', '9', '2022-11-24T10:00:10Z', '', 'type2']

not as:

['#datatype', 'string', 'long', 'dateTime:RFC3339', 'double', 'string']
['#group', 'false', 'false', 'false', 'false', 'true']
['#default', '_result', '', '', '', '']
['', 'result', 'table', '_time', '_value', '_field']
['', '', '0', '2022-11-24T10:00:10Z', '0.1', '_1_current_(mA)']
['', '', '1', '2022-11-24T10:00:10Z', '4', '_1_current_limit_(mA)']
['', '', '2', '2022-11-24T10:00:10Z', '1', '_1_voltage_(V)']
['', '', '3', '2022-11-24T10:00:10Z', '1', '_1_voltage_limit_(V)']
['', '', '4', '2022-11-24T10:00:10Z', '0', '_2_current_(mA)']
['', '', '5', '2022-11-24T10:00:10Z', '0', '_2_current_limit_(mA)']
['', '', '6', '2022-11-24T10:00:10Z', '0', '_2_voltage_(V)']
['', '', '7', '2022-11-24T10:00:10Z', '0', '_2_voltage_limit_(V)']
[]
[]
['#datatype', 'string', 'long', 'dateTime:RFC3339', 'string', 'string']
['#group', 'false', 'false', 'false', 'false', 'true']
['#default', '_result', '', '', '', '']
['', 'result', 'table', '_time', '_value', '_field']
['', '', '8', '2022-11-24T10:00:10Z', 'K', 'type']
['', '', '9', '2022-11-24T10:00:10Z', '', 'type2']
[]

Checklist

  • CHANGELOG.md updated
  • Rebased/mergeable
  • A test has been added if appropriate
  • pytest tests completes successfully
  • Commit messages are conventional
  • Sign CLA (if not already signed)

@bednar bednar marked this pull request as ready for review December 6, 2022 12:57
@bednar bednar requested a review from powersj December 6, 2022 12:57
Copy link
Contributor

@powersj powersj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current users probably have logic in place to avoid these lines, my only concern is if someone is counting on these being there to do some sort of parsing or counting. Overall though, this is an improvement over the current state.

@bednar
Copy link
Contributor Author

bednar commented Dec 7, 2022

I don't think there's anyone who relies on empty CSV row. From my POV it is a bug.

@bednar bednar merged commit 2d1fb2c into master Dec 7, 2022
@bednar bednar deleted the query_csv_empty_lines branch December 7, 2022 07:56
@bednar bednar added this to the 1.35.0 milestone Dec 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Query stops when encountering a field containing a string (query_csv)
2 participants