DSC to influxDB failing

[ghost]

On the collector side, when collection is done for:
dataset second_ld_vs_rcode dns Rcode:rcode SecondLD:second_ld replies-only max-cells=50;

Sometimes queries are received for null names:

<array name="second_ld_vs_rcode" dimensions="2" start_time="1556173500" stop_time="1556173560">
  <dimension number="1" type="Rcode"/>
  <dimension number="2" type="SecondLD"/>
  <data>
    <Rcode val="0">
      <SecondLD val="org.za" count="2"/>
      <SecondLD val="" count="2"/>
      <SecondLD val="amazonaws.com" count="1"/

which ends up after DSC-datatool as

second_ld_vs_rcode,server=servername,node=ns.coza.net.za,rcode=5,secondld= value=2 1556174460000000000

which chokes when fed into 'influx -import' because of the missing value @ "secondld="

dsc-datatool needs to add a null character when encountered, to prevent the influxdb import from breaking.

[jelu]

I believe I can fix this quite quickly but I'm having a hard time to find the time to do it as we are ramping up for our annual workshop :/

[ghost]

Yeah - done some looking at this - It seems to me that influxdb doesn't understand the concept of Null characters - so simply setting it to 'Null' as a string seems to work - not sure if that may cause problems going forward.

Naturally, if DSC-datatool could do this for us, life would be good.

[ghost]

Any updates on this? Anything I can do/supply to move this forward?

[jelu]

Just got back today from a lot of traveling, will look at this ASAP

[jelu]

Looking at this now, and a little searching tells that it might have worked before (somewhat):

You cannot search for nulls in InfluxDB <0.9. You will not be able to insert nulls in Influx >=0.9

It also looks like InfluxDB have much better support for quoting then it did when I first made this :)

[ghost]

Yeah - that was my reading - but then I only read the current influx stuff. My thinking is that if the field is blank, making it the string "Null" would suffice. My temporary work around was not to process xml files with fields like "", and I've been holding back on replacing them with "Null".

…

--Calvin

On 2019/05/30 08:41, Jerry Lundström wrote: Looking at this now, and a little searching tells that it might have worked before (somewhat): You cannot search for nulls in InfluxDB <0.9. You will not be able to insert nulls in Influx >=0.9 It also looks like InfluxDB have much better support for quoting then it did when I first made this :) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#31?email_source=notifications&email_token=AL55FZGNGPWK72YOREVN4MTPX5ZJRA5CNFSM4HI46SJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWRQLKY#issuecomment-497223083>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AL55FZBTMZJGTOAX5R2NR5LPX5ZJRANCNFSM4HI46SJA>.

[jelu]

That was a quick and easy fix, sorry that it took so long.

Can you test using the develop branch or do you need a release? If so, what dist?

[ghost]

As soon as i work out how to clone the development branch, I'll test.

[jelu]

git clone https://github.com/DNS-OARC/dsc-datatool.git
cd dsc-datatool
git checkout develop

[ghost]

looking good on my test stuff - throwing it at some live data - will revert

[ghost]

Right - done and dusted - adding you to my list of people I owe a beer/coffee to.

[jelu]

Great to hear :) I will make a release in the coming days.

If you want to support DNS-OARC and our software please consider becoming a member. There is also options to do one-time donation, let me know if so!