add capability to remove more than one "incomplete" burst from the end of a file

[rallen-usgs]

In the recent vector data, I have found there is often more than one incomplete burst at the end. I'm not totally sure why, but I think perhaps it's because the instrument is connected to while a burst is being collected, and then it starts to get a new burst. The dat2cdf code works fine, but the cdf2nc code gets hung up by this, probably because there is more than one burst that is not full. I'm not sure of the best way to handle this, but at the moment I've been editing the output files by hand, which I'm sure is not the best way. Happy to submit example vector files if that would help.

[ssuttles-usgs]

Might be a good candidate for using multi-indexing approach that can handle incomplete burst. It will fill missing data with Nan. Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: rmallen86 ***@***.***> Sent: Wednesday, January 17, 2024 10:19:20 AM To: USGS-CMG/stglib ***@***.***> Cc: Subscribed ***@***.***> Subject: [EXTERNAL] [USGS-CMG/stglib] add capability to remove more than one "incomplete" burst from the end of a file (Issue #180) This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding. In the recent vector data, I have found there is often more than one incomplete burst at the end. I'm not totally sure why, but I think perhaps it's because the instrument is connected to while a burst is being collected, and then it starts to get a new burst. The dat2cdf code works fine, but the cdf2nc code gets hung up by this, probably because there is more than one burst that is not full. I'm not sure of the best way to handle this, but at the moment I've been editing the output files by hand, which I'm sure is not the best way. Happy to submit example vector files if that would help. — Reply to this email directly, view it on GitHub<#180>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AJEPJ2HH5WNBJV3FGKEJINLYO7TXRAVCNFSM6AAAAABB6XDRWOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGA4DMNBRGUZTGMY>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[dnowacki-usgs]

Thanks Rachel. It would be good to see a file that has this issue. Assuming a full burst has 1024 samples, is it something like

burstnum   1    2    3    ...  N-2  N-1  N
numsamples 1024 1024 1024 ...  1024 965  1005

i.e. the length of the two truncated bursts is random?

[rallen-usgs]

Yes, I think so. I'm attaching here files from DMP23XL201vec, a vector that was used simply to log an acoustic backscatter sensor (AnalogInput1). You can see that bursts 531-534 are shorter than all the others; burst 534 is the last one.

I haven't attached gatt or config files - do you want those?

DMPXLB03.zip