No Input data error feedback

[marcus-aa]

The feedback from mlat-client is kind of strange when it doesn't like the input data.

Successful start, everything OK:

# mlat-client --input-type radarcape_gps

Mon Apr  3 03:04:15 2017 mlat-client 0.2.6 starting up
Mon Apr  3 03:04:15 2017 Connected to multilateration server at IP:port, handshaking
Mon Apr  3 03:04:16 2017 Server says: 

        In-development v2 server. Expect odd behaviour.

        The multilateration server source code is available under
        the terms of the Affero GPL (v3 or later). You may obtain
        a copy of this server's source code at the following
        location: ....
        
Mon Apr  3 03:04:16 2017 Handshake complete.
Mon Apr  3 03:04:16 2017   Compression:       zlib2
Mon Apr  3 03:04:16 2017   UDP transport:     disabled
Mon Apr  3 03:04:16 2017   Split sync:        disabled
Mon Apr  3 03:04:16 2017 Input connected to localhost:30005
Mon Apr  3 03:04:16 2017 Input format changed to RADARCAPE, 1000MHz clock

Start with auto, here it must not like the input data (only difference from above), but from the feedback "Lost Connection" it looks more like there is something wrong with the server. There is no "connected to localhost:30005", but it does connect.

# mlat-client --input-type auto 
Mon Apr  3 03:03:15 2017 mlat-client 0.2.6 starting up
Mon Apr  3 03:03:15 2017 Connected to multilateration server at IP:port, handshaking
Mon Apr  3 03:03:15 2017 Lost connection to IP:port
Mon Apr  3 03:03:15 2017 Reconnecting in 30.0 seconds
Mon Apr  3 03:03:45 2017 Connected to multilateration server at IP:port, handshaking
Mon Apr  3 03:03:45 2017 Lost connection to IP:port
Mon Apr  3 03:03:45 2017 Reconnecting in 30.0 seconds

Anyway, it's working great, just tricked me more than once thinking it was something else broken...

[mutability]

What is the datasource you're connecting to?

[mutability]

• can you get a pcap capture of a failing connection?

[marcus-aa]

Radarcape without status messages, as discussed here

http://discussions.flightaware.com/ads-b-flight-tracking-f21/mlat-coverage-t37889.html

[mutability]

So you are connecting to modesmixer then?

[marcus-aa]

Yes. and it works, just not with auto.

[mutability]

Right, so I reviewed the input path and it's fine; it'll always log about why it's disconnecting if the disconnection is initiated locally.

Then I took a closer look at your logs and it's not the input connection that's closing, it's the server connection. So probably the server is choking on something in the login message and you should look there.

[marcus-aa]

When it fails, the message

Input connected to localhost:30005

is missing.

[mutability]

That is because the input side connection is only started after the server connection handshake has completed (there's no point receiving input if you have no server connection, what would you do with the input)

It's probably something to do with the clock characteristics that are being reported; mlat-client has evolved in that area over time but mlat-server probably needs some work to catch up with it.

[marcus-aa]

Then no connection with "auto" would ever work, if it doesn't even look at what is offered and still gets rejected.

I have to check that!

[mutability]

Yes that sounds plausible.

[marcus-aa]

Sorry, I don't get it.

If auto-detect is going to work, it must check the input data first, detect it, and then tell the server what kind it is going to send. That is why I was expecting it to open local input first, then server connection.

Or is it supposed to change "on-the-fly", after the server connection is established?

[mutability]

It is supposed to change on the fly.

The most notable case is switching between RADARCAPE and RADARCAPE_EMULATED (free-running 1GHz clock not synchronized to GPS time) based on the Radarcape's GPS health.

[marcus-aa]

OK, It seems that auto doesn't work with the current mlat-server software, regardless of type of input data.

20170403 14:47:17.619  INFO     client               [x:y] Accepted new client connection
20170403 14:47:17.623  ERROR    client               [x:y] Exception handling client
Traceback (most recent call last):
  File "/home/adminuser/mlat-server/mlat/server/jsonclient.py", line 278, in handle_connection
    if not self.process_handshake(hs):
  File "/home/adminuser/mlat-server/mlat/server/jsonclient.py", line 367, in process_handshake
    connection_info=conn_info)
  File "/home/adminuser/mlat-server/mlat/server/coordinator.py", line 267, in new_receiver
    clock = clocksync.make_clock(clock_type)
  File "/home/adminuser/mlat-server/mlat/server/clocksync.py", line 63, in make_clock
    raise NotImplementedError
NotImplementedError
20170403 14:47:17.624  INFO     client               [x:y] Disconnected

[marcus-aa]

"The most notable case is switching between RADARCAPE and RADARCAPE_EMULATED"

Will this work since I have to specify the input format, and what should be used? radarcape or radarcape_gps ?

Is there any way to see from the server sync stats that a receiver is actually running with absolute sync from synchronized GPS time? It would be interesting to see if absolute synced receivers works (without adsb-ac-sync).

[mutability]

No, it doesn't really work with the current mlat-server. The client has been updated to notify the server, but the github server code hasn't been touched in quite a while and IIRC it doesn't understand the notifications. The version of the server I work on for FA looks quite different internally in terms of how clock sync is done, and that's the version that most of my mlat-client changes are aimed at.

If you're willing to make changes to mlat-server to do it properly I'm happy to take pull requests, but I don't really have the spare time & infrastructure to make those changes myself.

[marcus-aa]

OK, I understand that. We are very grateful for the current functionality and respect that time is in short supply. I wouldn't know where to start in your code so that is unfortunately not an option.

If you find more time and are interested to test the GPS-sync in a network, we could supply access to test infrastructure including suitable receivers, in a test area where no adsb flights exists for long periods of time....

[mutability]

If you're in radarcape_gps mode you should get GPS sync, in theory; see https://github.com/mutability/mlat-server/blob/master/mlat/server/clocknorm.py#L53 (radarcape_gps clocks have an epoch of gps_midnight, see make_clock in clocksync.py for how it's built)

The bit that doesn't work is changing the clock parameters midway through a connection IIRC, you need to deal with it like a clock reset and reinit the receiver's clock parameters. In the current code make_clock is only ever called when the receiver first connects, it needs to do something more around here: https://github.com/mutability/mlat-server/blob/master/mlat/server/coordinator.py#L330

It also needs to deal with the receiver potentially not knowing the clock parameters initially in the login message (should make sure that the receiver isn't used for anything until the parameters are known)

[marcus-aa]

I have previously looked around a bit in those areas of the code just for interest, will have a look again. I understand some of he basics but I'm not really a programmer and certainly not in object-oriented stuff...

Is this a problem only for hot-switching between modes? If the receivers stay gps-synced, would it work in the current code? I will have to do some monitoring to see if they un-synch regularly or not.

[mutability]

Yes just for switching. If you use radarcape_gps then the server won't notice the switches, so you'll see degradation if you do lose GPS, but it should continue to work somewhat until the undisciplined clock drifts too far.

[marcus-aa]

OK, thanks, I will try to optimize the network...