-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DSP structured output #94
Comments
I'll have to see what I can do in the future. It probably wouldn't be too difficult if the printing/payloads weren't already spread across a ton of files and so on, would it matter so much if it were direct console output, or would putting that output to a separate file be okay? Also, while you are here, let me ask you a few questions.
See You |
I don't mind any way, stdout can be captured, filtered and passed to other tools, files can be tailed in similar manner, so i think you can choose whatever way is the easiest (1) lrrpLRRP (MBXML documents) are appended to each other, and Document Type ID is single-byte (first byte), see https://github.com/OK-DMR/ok-dmrlib/blob/master/okdmr/tests/dmrlib/motorola/test_lrrp.py#L36 Also the first few bytes have meaning, see: https://github.com/OK-DMR/ok-dmrlib/blob/806050b582e5f5c19f04fe11ddc77ec2cd608ce4/okdmr/dmrlib/motorola/mbxml.py#L933
So you can check, if you have full pdu of allowed LRRP document (reading 2-4 bytes, checking doc-id and doc-len) However LRRP protocol PDU can come within data transmission, which can have some padding at the beginning (udp vs extracting un/confirmed dmr packet data transmission) also "tokens" are either global (mbxml global tokens) or protocol-specific (LRRP has specific tokens for request pdus and different for response pdus) and at last, if you have any LRRP data where tokens are not-detected correctly, please send them to me, either PR or issue for LRRP support on ok-dmrlib, because I have little to no data for some of the documents i have specifications for. (2) connect plusi have no information, specification or testing devices for connect plus, other than what ADK Overview and System Planner EA documents mention, sorry CSBK opcodes are here, but you already know probably, https://github.com/OK-DMR/ok-dmrlib/blob/master/okdmr/dmrlib/etsi/layer2/elements/csbk_opcodes.py#L10 if you mean MFID (Manufacturer Feature Set ID), see https://github.com/OK-DMR/ok-dmrlib/blob/master/okdmr/dmrlib/etsi/layer2/elements/feature_set_ids.py#L10 0x06 should be Trident Microsystems CSBK opcodes overlay (eg. TD_GRANT_MI and BS_Dwn_Act), so when detecting contents, you must take into account whether its MS or BS sourced "I'm not going there to die. I'm going to find out if I'm really alive" |
Also if you wanted to experiment, https://github.com/OK-DMR/dmr-kaitai *.ksy files could be translated into h/c/cpp form and you could use existing structures to parse incoming as whole, with kaitai struct runtime it should report when the PDU seems to be incomplete or help detect unknown fields But yeah, I tried to orient myself in your codebase, but i was unable to deduce which pointer/heap to use, so i asked directly, since you sometimes work with data in form of unicode string (eg. DMR_BS_DATA_SYNC "313333111331131131331131") instead of extracting the bytes into unified object and handling the pdu with set length (or through intermediary struct), it becomes messy and fragile down the line (i mean managing the pointers). I could not make much sense out of it. |
I'll have to review all that you just said on LRRP when I have more time, energy, and focus to go through it all again. Same for the dmr-kaitai stuff as well.
That's alright, it doesn't seem to be that there is any public information on it, other than what a few others have been able to figure out based on trial and error I presume, but I have a Connect Plus Voice Grant/Grant Update, so that's good enough for trunking.
Yeah, I agree, I'm not a big fan of the string stuff myself, but that code was in there from the original versions of dsd from dsd author, I have never changed it or attempted to do much with it. It works fine for what it does, with one exception, NXDN frame sync pattern. I have considered changing it a few times, but never wanted to get knee deep in fixing it and end up breaking a lot of other things. I'll also be the first to admit, I am not a programmer. I wasn't trained or taught how to program, didn't go to school for it, I just jumped in and keep going. A lot of things I do are probably technically wrong for any number of reasons, so don't judge my code too harshly, I just go by results. If it works, it works. There is also multiple hands in the source code, so a lot of different writing styles and preferences all mixed together. |
Easy there, not judging you for code style or abilities, but you orient better and i'm not fluent with c/c++, so if you think you can manage best with file output as you suggested, i'll be happy. You keep your community happy and alive, meaning there's nothing to look down on. Let me know, if I can help you somehow |
I see you've been working on other issues, is this one problematic? Do you need/want help? I can definitely try |
Sorry about that, just haven't had a chance to get started on it just yet. |
I had a question about that, do you want the second number on the line to be a hex character representing the data burst type? If so, what value would you give to a voice burst type? Another question, do the CACH and RC bursts need to be seperate, or can you just decode that from the relevant bytes presented. Here is a working mock up I made, in the zip file, along with a log from dsd-fme for comparison. You did say 33 bytes, so I'm assuming you didn't want the CACH leading off each line, just going straight to the beginning of the payload. Also, I used the characters d for data burst, f for an initial voice burst sync, and v for continuing voice bursts. |
Ignore that, no CACH is 33-bytes. Counting in dibits vs bytes lead to confusion on my end. |
I found a few errors in my initial mock up in the code, I think there may have been a few bytes short, and also I restructured it slightly, now its setup such that the first character is the slot, the second two characters are hex values representing data brusts, and the value of 10 (hex 0x10, dec 16) is designated for voice (bs), let me know if this format will work for you. I can tweak it some still if needed. I'm not sure still though how you want both RC and CACH though, do you want the completed CACH after its all put together but before deinterleave, etc? (for Slow Link Control I presume) or should I just add the CACH bytes to the beginning of each line before the payload? Same for RC or Single Burst Embedded (VC F)? Can't you just scrape that from the appropriate location on the line? |
Well man, good work, looks nice already, but data are currently not valid, from what i can tell TLDR
longeri will reference ETSI TS 361-1 V2.5.1 (2017-10) to you (PDF here: https://www.etsi.org/deliver/etsi_ts/102300_102399/10236101/02.05.01_60/ts_10236101v020501p.pdf )
Looking at your data, first 4 rows broken down
row no.3 (first valid sync) broken down:
Example valid data
|
Anyways, here is what I have so far, just a small segment, and also a zip file with 30 seconds worth of CACH and Traffic Payload. When you say [TS 1] [DATA TYPE], do you mean you want it in the file as:
or as
What I have right now is CACH and Voice/Data Payload
What I don't have right now is RC handling, at all. I had some code for it in the past but I scrapped it, it wasn't complete and I didn't have anything to test it with. Also, with MS sourced payloads, its only going to be the Voice and Data payloads, and no CACH. I've found that the data handling can be pretty rough on Simplex due to the way the demodulator works, I can get decent voice out of it, but if it isn't BS sourced audio, then data decoding/demodulation is not very good, and voice can have error as well, usually at the very start. |
Added some explanatory screenshots to my responses If you want, we can connect somewhere else, to solve things faster, telegram/signal/discord/irc/Element(previously Matrix)/... just let me know i'll send you my id on whatever communication platform you prefer I guess you have one copy of codebase and unless it's working, you don't want to commit/push commits to public, |
@lwvmobile to respond to your last template was overly verbose, yes please, this format, you already have, is probably the best:
ad RC, if you don't have handler for it already, we can skip or postpone RC bursts output, sometime later it might be re-implemented ad CACH see the screenshots above in #94 (comment) ad demodulation, i'd have to dive into the code, but if we have something already, in ok-dmrlib i have all the ETSI FEC features already, so if it can be repaired using FEC, i don't worry, if the burst is damaged too much, i will skip it's handling, no need to pre-filter your dsp output i'll pass the spike-cach.dsp to my tools and tonight we might see the traffic monitoring already working :)) |
I don't think any amount of FEC can fix the demodulator as it is. When there is no sync (MS skipping Time Slots) the noise causes issues for the demodulator and the dibit buffer (the aformentioned 131313 string nonsense that was written in originally). DSD was originally written with P25 in mind, they didn't envision an encoding type where the sync pattern is in the middle of the payload (which even I have to admit, is kind of annoying to place it there). I may fix the demodulator one day, but what I'd rather do is write or modify another program that can be the used as a demodulator (completely externalizing it) and use it to pass symbols directly to DSD-FME.
I prefer not to. No offense to you or anybody, but I prefer the disconnect and slow pace found here and in forums.
Yeah, maybe one day I'll learn more about 'feature branches' but to be honest, I'll push the code when I'm ready to push it. I'm working on a few other things as well at the moment, and when I feel its stable and been thoroughly tested, then I'll push it. I'll try to have it all wrapped up and pushed this upcoming weekend. I also have a 'lite' version to maintain and mirror changes to while maintaining the tweaks to make it work properly in 'windows' so between two different code bases, that's already twice as much as I'd ideally like to deal with. I also enjoy working when its at my own pace, and not me being rushed to get this out. Thanks. |
Sure, i get all your points and won't rush you. I think we have discussed the feature in-depth, i certainly don't have anything else to add to topic. |
Btw. i extended CSBK support in ok-dmrlib, from second data-set, aloha pdus look like this, just so you know, there is a lot of info, dsd+ does not expose
You can check TS 102 361-4 for C_ALOHA PDU (in my version ETSI TS 102 361-4 V1.10.1 (2019-08) its "Table 7.19: C_ALOHA PDU content") for meaning of the values, if they don't seem familiar to you More output, just for completenes
|
Yeah, I've got the manuals with all the PDUs in them from ETSI, I just haven't put them all in, just the ones I deem more beneficial than others when it comes to trunking support and who is talking to who, etc. |
Thought I'd catch up with you for a moment and see what you might know about Hytera XPT systems. Had a couple of samples I acquired recently and was investigating how the link control and csbks and how to properly decode them. A few LC and CSBKs that stood out to me include the following:
I've worked some support in for CSBK 0x3A (site status) and also the TLC for Voice 'Grant'? and Free Repeater, etc. Edit: Here is a second sample if it helps any. |
@smarek @lwvmobile Do you have any samples for DMR Data burst MS Sourced? I want to test those for my setup. Also a complete VOICE Superframe MS Sourced sample would be of great help! |
Here is a super short sample I got a couple years back of MS sourced Data, its not the best quality, however. I probably have others, but not about to dig around to try to find them. Not sure if I have any MS audio samples handy, can't seem to find any that I'd be at liberty to share, though. |
@lwvmobile Thanks for the sample, But I am unable to decode this one. Do you have the hex burst? That would be helpful |
Hi, thank you for this project, could you please comment on my feature proposal below?
Goal: Run option for dsd-fme so it dumps Normal (Standard), RC and CACH bursts in timeslot+burst-bytes format
Basically i'd want to process the on-air data via different application/project, and currently only parsed output and symbols dump (symbols file) are available.
So lets say "dsd-fme --raw-bursts -i 1R.wav" would either on-stdout or into-file dumped each DMR burst that went into processing by dmr handler methods, eg. like this
Eventually burst-type field could be [2-chars-ascii] if the feature was gradually rolled-in for more radio networks
The text was updated successfully, but these errors were encountered: