AHRS + ForeFlight 7.5 no longer works consistently

[egid]

There's a reddit thread on this, but I figured we'd be better off tracking an issue. The 7.5 upgrade seems to have broken AHRS, in that the Stratux device shows in the ForeFlight device list very, very rarely. I have had it show up, so it's not entirely busted. Just mostly. I've also seem a few seconds worth of AHRS.

This reproduces on 0.4b4 and 0.5b1.

Expected: AHRS, when enabled in webui (GPS/AHRS) and ForeFlight (Devices > Stratux > AHRS).

Actual: 'No Attitude Information', GPS-only SV, Stratux device not visible 99% of time.

I seem to be able to temporarily force the Stratux device to show for a second or two if I swap to the webui, toggle AHRS off and on, and return to ForeFlight.

[egid]

Here's what I sent off to ForeFlight:

Hi! I'm a tester for the open-source Stratux ADS-B In package, and it looks like the 7.5 ForeFlight update removes the ability for the Stratux to send AHRS data to ForeFlight. Two questions:

1. Was this a deliberate change to block Stratux from communicating with ForeFlight?
2. If so, can we work with you to restore compatibility?

Hopefully it's just a temporary thing - getting ADS-B data to more pilots cheaply is a huge win, and ForeFlight is the EFB of choice for many of us.

And here's the response from their support desk:

Thanks for your email. I will have to check with the developers to be sure, but I haven't heard any specific plans to modify the method that Stratux is using.

I also have not heard any plans to actively support Stratux, but I will pass your request to the right people.

I replied, and I'm waiting for a response.

Great! Thanks for checking into it. Even if there's no active support, there wouldn't be any reason ForeFlight would actively block a project like Stratux, is there?

I'll share any info I get back tomorrow. It does seem like they're at least vaguely aware that Stratux exists, based on the statement that he had 'not heard any plans to actively support Stratux'.

[RobertGary1]

Th reason would be to sell more Stratus devices. I'll be interested in hearing their response. If we can just get ahrs working on WingX maybe it doesn't matter.

-Robert

On Dec 9, 2015, at 11:18 PM, Eric Gideon notifications@github.com wrote:

Here's what I sent off to ForeFlight:

Hi! I'm a tester for the open-source Stratux ADS-B In package, and it looks like the 7.5 ForeFlight update removes the ability for the Stratux to send AHRS data to ForeFlight. Two questions:

Was this a deliberate change to block Stratux from communicating with ForeFlight?
If so, can we work with you to restore compatibility?
Hopefully it's just a temporary thing - getting ADS-B data to more pilots cheaply is a huge win, and ForeFlight is the EFB of choice for many of us.

And here's the response from their support desk:

Thanks for your email. I will have to check with the developers to be sure, but I haven't heard any specific plans to modify the method that Stratux is using.

I also have not heard any plans to actively support Stratux, but I will pass your request to the right people.

I replied, and I'm waiting for a response. I'll share any info I get back tomorrow.

Great! Thanks for checking into it. Even if there's no active support, there wouldn't be any reason ForeFlight would actively block a project like Stratux, is there?

—
Reply to this email directly or view it on GitHub.

[egid]

I think it's worth reserving judgment on that until they explicitly say, 'hey, this isn't a thing we support'. ForeFlight supports AHRS data from Dynon and Garmin panels, so I have a hard time believing they're cutting off support for open-source devices in an attempt to sell more Stratus boxen. Especially since the people hacking together rpi-based ADS-B units are hardly the sort to drop $1000 on a Stratus 2S.

[cyoung]

I'm not able to troubleshoot this for another few days, would anyone be able to try ahrs_replay and see if GPS/AHRS simulator works reliably with FF now?

[hclark]

I just tried last night, it seems FF shows both Stratux and FreeFlight in the green on startup, then the Stratux box shows "disconnected" and disappears. No AHRS (but shows up ok in the Stratux mgmnt. connection)
No AHRS in FlyQ as well, but I'm not sure if that was the case with 0.4r4 or not. This seems to have started after going to 0.5b1, but may be coincidental.
I'm going to go back to 0.4r4 tonight and see if the problem is still there.

[egid]

@cyoung - I'll see if I can get some logs tonight and try it out.

@hclark - that's exactly what I was seeing. Brief connection, then dropped.

[cyoung]

Just try this https://github.com/cyoung/stratux/tree/master/test-data/ahrs

[egid]

👍

[hclark]

Just loaded v0.4r4, same thing. Getting AHRS data to the STATUX config. page, but nothing to FF or FlyQ. The Stratux icon popped up in FF for a few seconds, then disappeared.
Maybe some change in the iOS?
Sorrry, I don't know what the "Just try this" link is supposed to do...

[hclark]

I also noticed a reference somewhere to FF not working with FSX/FSXFlight, so I just checked and it works fine, full AHRS display. So it appears it's not the FF app anyway.

[ghost]

This seems to be related to our use / abuse of ForeFlight's flight simulator support to display AHRS pitch, roll, and heading.

Stratux sends a heartbeat packet, traffic, weather, and simulated ownship information (GPS position / track / speed / pressure altitude) over port 4000, using the standard GDL90 format. This information is structured the same way as it is by FreeFlight Systems' wifi module; hence ForeFlight displays the basic Stratux as a "FreeFlight" device.

Notably, this approach does not include AHRS attitude. Instead, we turn to the flight simulator support documented above. To support desktop flight simulators, ForeFlight accepts simulated GPS, simulated traffic, and simulated attitude information over UDP port 49002 using the XGPS, XTraffic, and XATT tags. We use XATT to provide the pitch / roll / heading data.

Up through version 7.4.x, ForeFlight supported simultaneous display of port 4000 GDL90 and port 49002 flight simulator data. Starting with version 7.5, ForeFlight appears to suppress the data from port 49002 if it detects a GDL90 heartbeat on port 4000. You can demonstrate this by commenting out the heartbeat, traffic, and ownship functions in gen_gdl90.heartBeatSender(), and recompiling :

func heartBeatSender() {
    timer := time.NewTicker(1 * time.Second)
    timerMessageStats := time.NewTicker(5 * time.Second)
    for {
        select {
        case <-timer.C:
//          sendGDL90(makeHeartbeat(), false)
//          sendGDL90(makeStratuxHeartbeat(), false)
//              sendGDL90(makeTrafficReport())
//          makeOwnshipReport()
//          makeOwnshipGeometricAltitudeReport()
//          sendTrafficUpdates()
            updateStatus()
        case <-timerMessageStats.C:
            // Save a bit of CPU by not pruning the message log every 1 second.
            updateMessageStats()
        }
    }
}

This will suppress any of the port 4000 GDL90 messages from being transmitted -- i.e. the "FreeFlight" device effectively ceases to exist. All Stratux will send is the AHRS attitude data on port 49002. At that point, ForeFlight will recognize the Stratux AHRS device, and it will react as expected to pitch / roll inputs on the MPU6050 gyro.

Frankly, I'm surprised they haven't done this sooner. Having a device simultaneously accept simulated inputs from X-Plane / MSFS and a certified FreeFlight device is a potentially serious safety of flight issue: any device on the same WiFi network as the Stratux and iPad can multicast packets to create false GPS positions, false attitudes, and false traffic reports. The long-term solution would be officially-documented GDL90 extensions that could be used in conjunction with the ownship report.

[cyoung]

The long-term solution would be officially-documented GDL90 extensions that could be used in conjunction with the ownship report.

Which are those?

[ghost]

@cyoung -- that's the rub, isn't it? I certainly don't have them; I'm guessing by your comment that FF hasn't bought onto the 0x4c message format.

It's not unprecedented that FF would provide third-party access to display AHRS data (witness Garmin Flight Stream and Dynon SkyView). The question is whether they'd open an API for an open source project like this.

[Axtel4]

Not to convolute the issue, but this may explain why I have not been able to get the AHRS function to work with DroidEFB. I see the AHRS data in the Web UI, but it is not recognized in DroidEFB.

In stratux.conf there is a capability parameter in: {"Conn":null,"Ip":"","Port":43211,"Capability":X} Wouldn't setting capability to 4 send the AHRS data out in the GDL90 format and not as the FFSIM format?

Reference main/network.go:
48 const (
49 NETWORK_GDL90_STANDARD = 1
50 NETWORK_AHRS_FFSIM = 2
51 NETWORK_AHRS_GDL90 = 4

[monadical]

Sounds like a good followup question for @egid to ask FF. Are they indeed blcoking port 49002 when they have activity on port 4000. And most importantly, could they please be nice, pretty please, and let us know the protocol for sending AHRS data to port 4000 (or some special way to allow both ports again).

[skypuppy]

The only way to reach those guys is through their wallet.

On 12/10/2015 11:56 PM, monadical wrote:

Sounds like a good followup question for @egid
https://github.com/egid to ask FF. Are they indeed blcoking port
49002 when they have activity on port 4000. And most importantly,
could they please be nice, pretty please, and let us know the protocol
for sending AHRS data to port 4000 (or some special way to allow both
ports again).

—
Reply to this email directly or view it on GitHub
#138 (comment).

[RobertGary1]

Is the interface we need the same that Stratus uses with WingX? Could @hilton help with the ahrs protocol he's getting from Stratus?

-Roberr

On Dec 10, 2015, at 10:08 PM, skypuppy notifications@github.com wrote:

The only way to reach those guys is through their wallet.

On 12/10/2015 11:56 PM, monadical wrote:

Sounds like a good followup question for @egid
https://github.com/egid to ask FF. Are they indeed blcoking port
49002 when they have activity on port 4000. And most importantly,
could they please be nice, pretty please, and let us know the protocol
for sending AHRS data to port 4000 (or some special way to allow both
ports again).

—
Reply to this email directly or view it on GitHub
#138 (comment).

—
Reply to this email directly or view it on GitHub.

[hilton]

@RobertGary1 Looks like you mentioned the wrong person.

[egid]

Received this morning from Jason Miller, their CTO:

Thanks for the note - we have been working to clean up device connection logic over the last few releases to address a set of bugs folks have reported.

The code does a better job of filtering/prioritizing to a "valid" set of connections (typically just one device at a time) - for example, when you are talking to FreeFlight in the cockpit the app no longer accepts flight sim data input as that setup doesn't make "sense" (i.e. if we know you are doing real flying, we shouldn't be using sim data).

More importantly, the flight sim data feed doesn't provide the sanity/safety checks on the data that are critical for AHRS sources we use in-flight. It also doesn't support the various calibration features that are needed. We don't have AHRS support for the FreeFlight data feed - no plans to add that anytime soon as we are swamped with other enhancement work.

I'm following up and asking if they'll work with us when they start supporting FreeFlight AHRS data.

[hclark]

A well-crafted answer. Since no positional information on a portable device can be used for navigation, it seems irrelevant to be concerned about the source. Kind of puts the Bad Elf folks out on a limb.

[egid]

My reply:

Would you or one of your developers be interested in monitoring (or perhaps participating at) http://github.com/cyoung/stratux? As AHRS nears completion I imagine it could help test your eventual FreeFlight/GDL90 AHRS support, if nothing else. It'd be great to be able to provide a full solution on inexpensive hardware like this.

Once your support is built, do you see any hurdles for the Stratux project to get data into ForeFlight? Or would the FreeFlight support be source-agnostic?

[dbarrington1]

How does the Stratus give AHRA data to Foreflight?

Sent from my iPhone

On Dec 11, 2015, at 3:08 PM, Eric Gideon notifications@github.com wrote:

Thanks for the note - we have been working to clean up device connection logic over the last few releases to address a set of bugs folks have reported.

The code does a better job of filtering/prioritizing to a "valid" set of connections (typically just one device at a time) - for example, when you are talking to FreeFlight in the cockpit the app no longer accepts flight sim data input as that setup doesn't make "sense" (i.e. if we know you are doing real flying, we shouldn't be using sim data).

More importantly, the flight sim data feed doesn't provide the sanity/safety checks on the data that are critical for AHRS sources we use in-flight. It also doesn't support the various calibration features that are needed. We don't have AHRS support for the FreeFlight data feed - no plans to add that anytime soon as we are swamped with other enhancement work.

I'm following up and asking if they'll work with us when they start supporting FreeFlight AHRS data.

—
Reply to this email directly or view it on GitHub.

[egid]

@hclark I interpreted his answer as saying "we're going to officially support non-Stratus AHRS, but not soon". I can see it being interpreted otherwise, though. I'm not sure where it puts Bad Elf, aside from unlikely to be funded anyway.

@dbarrington1 Stratus only works with ForeFlight, so presumably it's a proprietary solution. As far as I understand it also doesn't use GDL90. Then again, the better question is perhaps 'how do Dynon and Garmin give AHRS data to ForeFlight?' :\

[hclark]

Info. from Seattle Avionics...
I’m not a developer but I vaguely remember a discussion about which UDP ports to use for FlyQ and which for ForeFlight. The general idea was to disable sending data to both the “ForeFlight” port and the one used by basically everyone else. I think you want to use the ports that the Clarity system uses. it has both AHRS and ADS-B so is a good model to use. I don’t know which port(s) that is offhand but you should be able to find something like this in the Stratux group. or just ask Chris.

[hiltongoldstein]

Stratux AHRS support has been added to WingX Pro7 - this version will be released within days.

Hilton

[egid]

Still waiting to hear back from ForeFlight after my last response.

[jamesshannon]

"Proprietary" or not, has anybody looked at the communication between Stratus and Foreflight?

It's got to follow some basic protocols (UDP or TCP to start).

[cyoung]

Well, Seattle Avionics FlyQ supports it. So I guess they must have. Are there any other EFBs that support it?

[hclark]

Someone explained it earlier; the ADS-B signals get sent to one specific
port, whereas the AHRS signals get sent to a different port. Apparently FF
teamed up with FSX so that the sim. could send AHRS data to FF when using
FF with FSX.
Apparently FF decided it wasn't a good idea to send the data during a real
flight, so if FF is receiving ADS-B data over that specific port, the other
port is blocked.
Sounds a bit hokey to me, as none of this can be used for navigation
anyway, it's all situation awareness only.

On Thu, Dec 31, 2015 at 11:54 AM, James Shannon notifications@github.com
wrote:

"Proprietary" or not, has anybody looked at the communication between
Stratus and Foreflight?

It's got to follow some basic protocols (UDP or TCP to start).

—
Reply to this email directly or view it on GitHub
#138 (comment).

[hclark]

? I can't receive AHRS with my FlyQ - I asked Steve Podrachik about it and
he referred me to you.

On Thu, Dec 31, 2015 at 12:00 PM, cyoung notifications@github.com wrote:

Well, Seattle Avionics FlyQ supports it. So I guess they must have. Are
there any other EFBs that support it?

—
Reply to this email directly or view it on GitHub
#138 (comment).

[jamesshannon]

@hclark - My understanding is that that's how stratux works (or worked before Foreflight "cleaned up" their logic). However, Stratus 2 sends AHRS and it's not clear to me how it does so. If it used the same flight sim port "hack" as the stratux does, then it would presumably have stopped working recently as well.

Since it didn't, it stands to reason that stratus uses some other communication.

Even if FlyQ were to support the stratus, it doesn't necessarily mean that FlyQ is using the same protocol as Foreflight -- it could be that the Stratus has a foreflight-specific protocol, and also branched out to something that FlyQ supports.

Anybody have a Stratus and FF and is handy-enough with wireshark or charles to log a session?

[1-prop-head]

Hope this helps:
The attached file is a WireShark capture of a brief Stratux2-ForeFlight
v7.5.1 Synthetic vision session ran at ground level.

On 12/10/2015 06:48 AM, cyoung wrote:

I'm not able to troubleshoot this for another few days, would anyone
be able to try ahrs_replay and see if GPS/AHRS simulator works
reliably with FF now?

—
Reply to this email directly or view it on GitHub
#138 (comment).

[ssokol]

@1-prop-head - don't see an attachment. If github doesn't like the upload you might try linking to the file on dropbox

[1-prop-head]

Lets try this capture file from WireShark again ForeFlight v7.5.1 Stratus2 (not 2s) AHRS active
WireSharkCapture.zip

[cyoung]

Ground level, no towers I assume?

[cyoung]

Suppose you took the capture at around 0440Z?

[jamesshannon]

There was a network started by the stratus, the ipad with FF, and a computer (with wireshark)? Presumably the computer joined the network second (before the ipad)?

So the stratus is 10.26.36.1. I see a lot of broadcast UDP traffic going out on 41501, 41502, and 41504. (Flight sim support described earlier is on 49002, FWIW.)

I see a lot of random DNS requests (e.g., msftncsi.com) from 10.26.36.2, so I suspect that's some random background stuff on the laptop.

10.26.36.3 (presumably the ipad) only sends out 3 messages (all UDP): "i want to play ffm udp". Weird, but probably not relevant.

The .1 (stratus traffic) is beyond me. Lots of messages that start with 0x c2 53.

There are a handful of messages with that have a series of integers that are similar (but not exactly the same as):

501010-000066R06501010-000065R00150500-00000

Does that look like ahrs data at all?

[cyoung]

We were reading the ffm ("FF" Mobile) message before: c38c384. FF sends this message and a few others when it goes out of context, iPad wakes up, or app just starts.

[1-prop-head]

On 01/01/2016 01:15 PM, cyoung wrote:

Ground level, no towers I assume?

—
Reply to this email directly or view it on GitHub
#138 (comment).

yes no towers

[1-prop-head]

It was at 4:40 and it was done by a window in the house. The laptop tried to connect first but seemed to get rejected but still WireShark was seeing traffic so I connected ForeFlight and brought up the synthetic vision on the map page. Then I picked up the Stratus2 and tilted ti a bit and watched the iPad respond as expected. At that point I saved the capture.

On Jan 1, 2016, at 2:06 PM, cyoung notifications@github.com wrote:

We were reading the ffm ("FF" Mobile) message before: c38c384. FF sends this message and a few others when it goes out of context, iPad wakes up, or app just starts.

—
Reply to this email directly or view it on GitHub.

[1-prop-head]

I can record a real flight in the next couple of days and send you the WireShark capture file of the Stratus2-ForeFlight wifi comm. Perhaps with a matching Stratux log talking to a second iPad. Think about what you want to look at and I will set it up

On Jan 1, 2016, at 1:40 PM, cyoung notifications@github.com wrote:

Suppose you took the capture at around 0440Z?

—
Reply to this email directly or view it on GitHub.

[braddward]

Just a thought - since X-Flight will broadcast to Foreflight, would it be a lot easier to do a WireShark capture of that for the synthetic vision portion?

[1-prop-head]

Yet another WireShark capture file. This on has two different files the kmgyklpr2 file was done north east bound at 5000 feet using earlier firmware on the Stratus2 the klprI74 was done south west bound at 4500 feet and Stratux2 was updated just before take off so stratus had the latest firmware. ForeFlight was current and active and synthetic vision was working during these captures.
Stratus2capture.zip

[cyoung]

There's not a whole lot to be done here - Tyson from ForeFlight has stated that this was done for liability concerns on their end. It appears (to me) that they have taken it upon themselves to endorse and guarantee every bit of data input into and displayed from their app.

In either case, @hiltong has put forth effort to make sure that stratux and WingX are working together well - so if you want that, then that is currently the best option.

[hclark]

So Chris;
FF won't release the port for you to use with Stratux?
Has Steve P. ever responded for request on using Stratux w/FlyQ?

On Sun, Jan 10, 2016 at 4:44 PM, cyoung notifications@github.com wrote:

There's not a whole lot to be done here - Tyson from ForeFlight has stated
that this was done for liability concerns on their end. It appears (to me)
that they have taken it upon themselves to endorse and guarantee every bit
of data input into and displayed from their app.

In either case, @hiltong https://github.com/HiltonG has put forth
effort to make sure that stratux and WingX are working together well - so
if you want that, then that is currently the best option.

—
Reply to this email directly or view it on GitHub
#138 (comment).

[cyoung]

No, they won't.

Steve P is responsive and willing to work on compatibility - we need to work on it.

[hclark]

Glad to hear Steve P. is amenable; I have a lifetime subscription to FlyQ
:^).
Unfortunate FF won't cooperate, particularly since none of these apps can
be used for navigation anyway.

On Sun, Jan 10, 2016 at 6:46 PM, cyoung notifications@github.com wrote:

No, they won't.

Steve P is responsive and willing to work on compatibility - we need to
work on it.

—
Reply to this email directly or view it on GitHub
#138 (comment).

[cyoung]

Nothing we can do about this - use WingX or disable port 4000 if you want to test AHRS with FF.