Improve Taranis Latency #4640
Comments
|
Sending twice as much should not be a problem. FrTX is sending MUCH more frames to the XJT module and it works fine. And how to deal with dual XJT modules? Is there a heartbeat from the second module also even with telemetry fully disabled? |
|
If we do something it would be nice to do it right, since we already already schedule calculations based on when we transmit it would be cleaner and more optimal to align that to heartbeat than to spam the module with less recently updated data or increase CPU consumption with more frequent calculations than needed. But there's care to be taken for spurious heartbeat signals as in the dual XJT case. I don't know if Mike has hardened his implementation for this case, but if not done properly receiving a heartbeat pulse while processing the previous could cause problems. On the other hand, most of this latency nonsense is only there as a marketing con and I think we've got much more important / interesting / actually useful stuff to do than play that game and by that help "justify" all that "omg you absolutely NEED to shave off those couple of ms! Get our new XXX stuff!" BS. |
|
I am very interested in a total latency test from stick input to FC reception. If that is close to the latency differences it makes sense to optimize it. For planes nobody cares... :) 150msec added delay is not noticeable by an experienced flier.... |
|
And why do you think that it matters when using the FC!? What is important in such case it the local loop latency - ie accelerometer - CPU - servo outputs - ESC - motor response time is what it matters for fast multicopters and not so much the radio link latency. |
|
That. Many people don't understand that and say "we've now got FCs that look at xx kHz, the radio link has to do the same!" but no. It's still a mechanical thing, and while local loop speed certainly improves stability and response (up to a point) the response to controls still has inertia to overcome and that already represents more than RC link latency. |
|
True but the loops are mainly 1khz or better so 1msec between calculations. This is not true latency but it is what counts for most, and i can agree. With all esc / motor optimizations latency in the quad is getting lower and lower. With the small powerfull brushless quads inertia is overcome with raw power. |
|
Another popular and similar marketing BS is the stick ADC resolution. |
|
If Taranis is in 9ms mode (D16 1-8ch), but its really 20~24ms, heck yes it matters (well not to most of the plane guys). Going from D4R to X4R was night and day, next was 1-16ch to 1-8ch, another noticeable improvement (for good quad pilots), why not drop 1-9ms off, it's not a few ms, its 50% less. It's already been done on ersky9x, why not do it to OpenTx? |
|
No, it is not 50% less of total probably, so wee need to know total first. But it could be noticable... |
|
I just had a look at some blackbox logs... the time between the quad receives an order to move and the time it actually starts to move (motor response is near instant thanks to fast loop, BUT the mass has inertia and takes time to respond as mentioned previously) is 70ms. 9ms (at best! average gain would only be half of that!) is negligible in comparison.
But from D8 (with PPM I guess) to D16 (SBUS) you shaved off about 10 times more than those average 4.5ms.
Go ahead and do it, no prob! Anyone interested is welcome. |
|
"Many people don't understand that and say "we've now got FCs that look at xx kHz, the radio link has to do the same!" but no. It's still a mechanical thing, and while local loop speed certainly improves stability and response (up to a point) the response to controls still has inertia to overcome and that already represents more than RC link latency." That's not true at all, I can move the stick from center to full in 40ms, but it only takes a few ms to move it a little (like corrections), say 5 or 10ms, then the sampling of the stick to output of the Rx is ~24ms, 5 times longer than my stick movement took. I am now over 50 and I can feel the difference in lower latency. "On the other hand, most of this latency nonsense is only there as a marketing con and I think we've got much more important / interesting / actually useful stuff to do than play that game and by that help "justify" all that "omg you absolutely NEED to shave off those couple of ms! Get our new XXX stuff!" BS." Have you tried PPM and SBUS, can you tell a difference? By staying BS, clearly you're not in the high preformance quad racing and that is fine, this stuff does make a difference to us. Why did Spek spend a lot of time to minimize latency from stick to Rx output on the first 4 channels, because top pilots can tell a difference. We don't need it on all the channels or the "second" ext module. We drop the X4R from 16 to 8ch to get a noticable improvement. The new racing quads are pushing the limits of control, a lot of us laughed when they said we can do a flight control looptime of 1KHz, but it turns out it was much better than 300Hz, then 4K, 8K, 16K and now 32K, every one has made an improvement. Happy Flying! |
|
My quad from hover to 1400d/p/s is ~60ms, most racers use about half that roll rate, and corrections are a part of that, so under 20ms.
|
|
"It's still a mechanical thing, and while local loop speed certainly improves stability and response (up to a point) the response to controls still has inertia to overcome and that already represents more than RC link latency." Taranis 20+ms, quad under 10ms (EDIT: or less than 5ms, see below). "People's capability to anticipate is 10 times more important than latency. Problem is BS marketing is playing the "get the fastest gear so you can do like the pros!" which is completely incorrect. The pro can do well with 3 year old gear, " Anticipation is only part of flying/racing. After you anticipate and then make the move, then the small corrections are needed, this is not anticipation and where lower latency is king, it makes you feel more connected to the vehicle. With lower latency, you have to anticipate less ahead of time which is better for everyone, even though the casual pilots won't notice. EDIT: The main dev for betaflight posted this: With this data it looks like the Taranis latency is about 4 times slower than the quad reaction time. Since flight control looptimes are generally 32~128us, this is not a factor. We are not in sales or marketing, this is real data. |
|
I don't think my code needs any changing if you have two XJT modules. You can't sync. to both and as the heartbeat signal is open collector, wire-ored, you can't pick out a specific heartbeat. Mike. |
|
Can you point me at your implementation? I will take a look how feasible it is to port this to opentx. |
|
Ersky9x sources are here: https://github.com/MikeBland/mbtx In pulses_driver look at TIM1_CC_IRQHandler. The EXTI9_5_IRQHandler() must run at the highest priority so it has no latency itself in reading the timer. Mike. |
|
This guy (GadgetMart) pretty much hates the Taranis, I quote him in the rcg Taranis thread: "it is supposed to have a low latency rf module. Up to 60% faster. Like the Horus." |
|
@Ozzy33 planing to, when time permits, but that might not be anytime soon. So if you really need the feature now, don't bet on me. That low latency rf module is probably nothing substantiated. I am not aware of any differences in terms of latency in the Taranis/Horus modules. And all Taranis (X9D, X9D+, X7D) modules run the same firmware. |
|
Thanks for the reply, I am not looking for anything real soon, thanks for looking into this. |
|
This guy has been on my ignore list for quite a while, he's just a troll. Most of his posts are intentional provocation or unreasonable comments often backed with wrong "info". |
|
BTW, looks like your data was missed by some of us (you edited your posts instead of adding new ones, so no notification...). It does show that the improvement may be worthwile indeed and was what we wanted before considering time spent on that to be an interesting investment. |
|
Thanks! Yea, I am a total noob at github, I am not even sure how to correctly quote someone's post, doh! Thanks again. |
|
I have been following this idea for a while and here are a few comments and notes in case it is helpful to whoever tries to implement this... The Horus IXJT is different from the XJT and FrSky have stated that it has lower latency and probably is "faster". Apparently FrSkyOS sends updates to the IXJT more frequently than every 9ms and that may account for their claim to have lower latency on the Horus. They don't need to sync if the FrSky OS sends PXX data a lot faster than 9ms - say 2x 8ch PXX packets per 9ms frame - because the IXJT will always have the latest data when it sends..either lo8 or hi8. I suspect OpenTX 2.2 sends PXX packets every 9ms on the Horus and so gets more or less the same varying latency as the X9D+. According to Mike B's findings, the X9D+ XJT (Q-X7 is likely the same) apparently cannot tolerate PXX updates twice as fast and it dropped transmissions when he tried. Sync is a good solution in that case. One issue with it is that the heatbeat is every 9ms and does not indicate if the XJT transmitted lo8 or hi8. To get the best latency you have to restrict the PXX interface to sending only 8 channels. Then the XJT will either optimize to sending lo8 every 9ms (it is widely believed it does) or if it does not optimize then at least the lo8 data will be fresh each time the lo8 packet is sent every 18ms. If you send 16 channels in 2 PXX alternating packets, one every 9ms, then you risk being out of phase and then the latest data has to consistently wait > 9ms before the XJT will send it. EDIT: This would still be an improvement over the current situation where the PXX transfer and XJT transmissions drift in an out of phase causing the latency to vary over time From what I can gather the D16 protocol does allow optimization to just sent the lo8. I haven't seen anything definitive that indicates it actually does this although there are plenty of anecdotes. I'm pretty sure that the channels are ordered 1-8 and 9-16 in an array but not named. I think there is a flag or something that says if the array is lo8 or hi8. EDIT: protocol information can be gleaned from https://github.com/pascallanger/DIY-Multiprotocol-TX-Module Another point is that the external module heatbeat pin is apparently used by the "Linker" wireless trainer feature so presumably sync'ing the Module and using the "Linker" wireless trainer feature will need to be mutually exclusive either via a build option or via some other configuration. Presumably Mike B has handled this in er9X so the same solution might work for OpenTX. The Horus IXJT cannot run exactly the same firmware as the X9D XJT/ Q-X7 XJT/ XJT because FrSky made fixes to it on the Horus to i) allow both antennas to be used for TX diversity and ii) to fix some kind of lockout that was causing RX failsafes. The original XJT does not have an antenna switch and as far as I am aware has not had that lockout problem. GadgetMart is likely mistaken in suggesting that the X9D+ SE has an IXJT. I have not seen that stated anywhere else. Without OpenTX being changed to send the PXX packets either in sync or twice as fast there probably no significant improvement so why would FrSky go to the expense and trouble? Also technically they may have to re-certify with the FCC if they put a new TX module in the Taranis case. Everything else on the SE is just mechanical assembly stuff. Mike's original post...with note about the heatbeat pin and Linker Mike's findings... A discussion of the phasing issue Edit: 4/27/2018: fixed typo |
|
FrOS updates the external XJT much faster also then OpenTX ;) The module can handle it :) |
|
That's exactly what he said above... The fact that nobody knows exactly what the XJT does is sure also putting off from making much effort there. Many things would need to be tried and measurement tools developed to really be sure the right thing is done. |
|
And also at least one other device is affected, but how many others out there would be too ? There are A LOT of OpenTx users, with extremely varying needs and usage, that's need to be factored in too. |
|
@MikeBland I started looking into implementing this on the Taranis. Since we already have a dedicated timer for the heartbeat/Trainer pin. Is there reason not to start this timer from the xjt output timer (with the stm32 timer sync support) and use the already existing code that captures trainer inputs to parse the heartbeat signal instead of the highest priority interrupt? Or am I overlooking something. |
|
Planning is already not easy when it is your main job, and you can therefore estimate you'll have 8h/day to work on that. But thats not the case here, we do that in addition to our real job. I can think next week I'll have 4 evenings where I commit 3 hours, until I'm sent on business trip, and all that is gone outside of my control. But more importantly, timing is in FrSky hands. We do our best to have software ready based on what they tell about product availability (and more than once we have learned product where hitting market when users start telling simply because FrSky forgot/didn't bother to tell us). Usually we are doing ok with it (x7/xlite), sometime not at all (Horus was beta for month, R9 Flex was not great on our side (yes, that's an understatement)). Because we think this is critical, we are doing special efforts and prioritising this highest in our OpenTx todo list, in a hope to be ready when FrSky will be. To be brutally honest, they have given us (unfinished) specs about what they are working on (which looks exiting), rough explanation on scope, but absolutely no info on timeline. If you want info about timeline, ask FrSky. The good news here is that they have involved us earlier than usual in their product cycle, so we hope to be ready when they are, or not too late afterwards |
|
Here's something that I hope does get addressed. This is a blackbox log from a quadcopter running betaflight 4.0. The screen contains one second of data logged at 2khz. The quad makes a 400 degree/second co-ordinated left turn. During the onset of the roll, RC setpoint rate to actual turn rate delay is approximately 8ms. RC rate and actual gyro rate are overlaid on the top panel. I am using an R-XSR in SBus D16 8 channel mode, Taranis, hall effect gimbals. The quad was flown within 20m of the transmitter, line of sight, with good RSSI. The lower panel contains one trace. It shows the step change in RC setpoint for each packet as it arrives. ie the change in stick position as reported for each packet. Note that the packets do not arrive at regular intervals. Nor, despite the stick movements being smooth, does the data represent a smooth stick movement. It appears to frequently indicate zero movement. I think this is actually due to sending the same data value twice in succession. This behaviour is quite common at relatively low rates of stick movement. The timing between changed data packets is not constant at all. Frequently data arrives in 10ms, 8ms pairs. The alternating step / zero / step / zero data stream plays havoc with current quadcopter code, which relies on regular smooth differential steps. Motors get hot when there are sudden ups and downs in motor control signals. Since a quadcopter can readily change rotational velocity within 10ms, any false 'flat' period - any flat period of 15ms or more - causes the quad to actively stop the requested rotation by activating the counter props. This slows down the turn and wastes energy. To put it bluntly, I'm yet to see a vaguely acceptably smooth set of packet by packet step changes on any of my R-XSR or XSR quads. While we can filter those steps to some degree, a strong enough filter causes significant delay. I suspect duplicate data packets were sent to improve reception distances in the event of packet loss. Drone racers are far more concerned about delays and reliable, smooth, per-packet transmissions that accurately reflect true stick movement, than about range. Please do all you can to not only improve latency, but also please put an end to the sending of the same data value twice in a row just because the rate of change is relatively small.
|
|
That would need to be said to FrSky, as we do not do any of that in OpenTX (It is possible that the module does that, but only FrSky would know). Since they are redesigning their protocol right now (with massive improvements in every sectors), it is probably a good idea to inform them earlier rather than later. I will point them to your input, but i think they value customers imputs higher than ours (and rightfully so) |
|
@ctz, frsky announced new features and improved rf protocol, maybe post it over there, as getting any feedback from frsky never happened wherever I posted (about latency or any other issue/feature, it would of been nice to even get a no answer, I only got ignored) |
|
Hi @3djc, I'm not quite sure why would you state this:
There is a number of bench tests have been done by some active users here that prove that the ersky9x has got x2 better latency comparing to OpenTX. As far as I know, performance of the ersky9x FW is more or less acceptable and does not cause spikes and other negative effects on some internal functions of Betaflight that Chris stated above, could you please confirm this @ctzsnooze? |
|
We clearly do not do any kind of channel prioritisation based on quantity of changes (or anything else for that matter). If this is happening as suggested above, it must be happening in the module. As for latency itself, as said earlier, it will be addressed in 2.3 |
There is no such thing. But people tend to forget that RF transmission isn't perfect, and you'll regularly have dropped packets. A dropped packet is... well, no new data for one cycle, aka last value is held or "duplicated". If you want to smooth that out you have no choice but to do it in the FC. And it will obviously add latency. |
This is definitely not the case of dropped packets. Something deliberately sending duplicate data even if RX and TX are very close to each other. |
|
Dropped packets will happen all the time regardless of distance. I'd expect probably 5% to be lost just about any time. |
|
Here we are talking to have duplicates 100% of time, every second frame is the same. |
|
While interesting, this is somewhat a waste a time, since we have been redesigning the whole process from ground up for OpenTX 2.3, There is probably not a single line of code left from 2.2.x as far as pulses are concerned, and we think FrSky is doing something similar also |
|
@3djc, is awesome! Just keep in mind that we are here and willing to help/test. |
|
Any updates, especially on the latency with external (e.g. multiprotocol) modules? |
|
For Multi, I think @schwabe said something like OpenTx part is done, but it would require changes in multi that have not being merge, or soemthing like that (I really do not know, I just seem to recall that). For ACCESS, we are runing the mixer (reading postions and making the channels computation) every 4ms, starting exactly 1ms before pulse are sent to module. Those are Xlite S/Pro data, the only one with ACCESS currently. The frequency might change a litlle with other modules, but it will still be synchronized, 1ms before sending pulses. |
|
So you're now properly syncing to the clock of the external/internal modules, eliminating the latency variation? (The module theoretically always has 1ms + transmission delay old data to work with, at every potential RF transmission cycle.) And with the new FrSky firmware what changes is that you simply send at a higher rate? (Whether the RF transmission is actually done every 4 ms is a different question.) |
|
No, ACCESS module do not send anymore heartbeat or anything of that nature. The dialog is 100% different from good old XJT, that been transport mechanisme (full high speed serial) or the protocol used. We set the tempo, the module follows the beat, if I may say |
|
@xnoreq see pascallanger/DIY-Multiprotocol-TX-Module#107 It never got merged by hpnuts |
|
And given it was never merged on multi a couple of years later it's now completely outdated at least on OpenTX side and would have to be redone from scratch. |
|
The new FrSky X9 Lite mentions "High-speed module digital interface". Do any of the changes improve latency with R9 modules as well? |
|
On capable hardware, R9 modules should get 4ms too. Given the only announced ACCESS module are the xlite Prop (ISRM-PRO)/ X9 Lite (ISRM-N) ones , impossible to know about the other ones |
|
We where under the impression that FrSky would get ACCESS right and that HB sync would not be needed. Of course, it turns out to be a mistake, we have therefore added heartbeat sync to all radio (ACCST and ACCESS) in 2.3. Thx MikeBland for is work on that. We have also reworked the timing of mixer runs, which now is run right before sending pulse. Both changes result in great improvement in latency (both raw latency and jitter) for existing and new radios |
|
We did a pr for it : #6482 |
|
Good job guys. I've done a bench test for OpenTX 2.3. It shows some significant improvements in latency. I've used X7, R-XSR, D16 (8ch mode). It is now on par with Ersky9x: https://docs.google.com/spreadsheets/d/1cFSzIh9LWjcdg4LolqD-gMdTAGd3qDqY6YDe6nbOjGY/edit?usp=sharing |
|
So about 12.5 ms latency with the X Lite Pro and RXSR with access. would love to see how the crossfire is doing and Futaba. |
|
It's actually a bit lower now with the 1.1.3 access module/receiver firmware. |
|
Yea I so the notes and was thinking it may be lower, super nice that you confirm it |
and others
It has been said the Spektrum system has less latency in many threads and on YT.
https://www.youtube.com/watch?v=_rxAPlYr9FQ
There is a 0-9ms latency inconsistency due to the OpenTx OS is not synced to the XJT heartbeat signal, Mike B has done this Tx OS to RF module syncing on erSky9x, should be capable doable in OpenTx.
I found this post by Mike Blandford:
I've done a very specific latency test using a Taranis plus (internal XJT) and a X8R (both using non-EU firmware). This is a test of the FrSky latency leaving out any latency in openTx/ersky9x. The protocol was set to PXX/D16.
I created a very specific pulse sequence on channel 6, and generated a 'scope signal as the PXX frame is created just before it is sent to the XJT. I then triggered the 'scope from the change in the output pulse from the X8R.
I noticed the X8R output pulses were synchronised to the XJT heartbeat pulse.
As the 9mS timing on the radio is just very slightly different to that of the XJT, the time between the PXX signal and the X8R output pulse slowly reduced to a minimum of about 10.4mS, at which point it jumped out to 28.4mS and proceeded to slowly reduce back down to 10.4mS.
Every 9mS, the XJT sends 8 channels, alternately channels 1-8, then 9-16, so channels 1-8 are only sent every 18mS, which explains the jump of 18mS from 10.4mS to 28.4mS.
By synchronising to the heartbeat signal, it should be possible to fix the latency to either about 11mS or 20mS, depending on the phasing between channels 1-8 and channels 9-16.
Since it takes around 3.5 mS to send a complete PXX frame, it would be possible to send all 16 channels within a 9mS frame time. Whether this would reduce the latency is subject to test.
My investigations continue!
Mike.
Edit: I added something to synchronise to the heartbeat signal. The result is the latency fixed at either 11.2mS or 20.2mS.
I did try sending the PXX data twice as often, but the result is the Rx lost signal at times.
ANOTHER POST:
PXX sends 8 channels every 9mS to the XJT.
The XJT also sends a "heartbeat" signal. I use this in ersky9x to synchronise the PXX data to the XJT to minimise latency, otherwise you can get a 'random' extra latency of 0 to 9mS. I'm not sure that openTx uses this signal.
The text was updated successfully, but these errors were encountered: