Forward URL - loosing data when processing over 30 per second

[quade1664]

I've noticed a performance issue when forwarding lots of data per second
I've can see all the data in the DB, but not all of it makes it to the other end after being forwarded.

As discussed here
https://www.traccar.org/forums/topic/forward-url-is-there-a-limitperformance-tuning-that-can-be-done/

I would like to suggest an alternative solution of using MSMQ or RABBITMQ to drop the messages, therefore bypassing the performance limitation.

I've been looking into this today, but would like some help on the architecture side, instead of a denied pull request, or just a poorly performant integration.

I'm suggesting the following
Add the config flags
Forward.MessageQueueEnabled (true, false)
Forward.MessageQueueType (MSMQ, RABBIT etc)
Forward.MessageQueueName (name of the queue goes here)

then in this file:
https://github.com/traccar/traccar/blob/master/src/org/traccar/WebDataHandler.java

Edit this method:
protected Position handlePosition(Position position) {}
And have an IF statement

if(Forward.MessageQueueEnabled){
//write forwarded message to queue.
}

Firstly does the above sound like an ok solution?

Next up is the type of Queue to use
I'm using microsoft message queue, however, thats mainly as i'm doing mostly C#, and I'm not sure how well Java plays with this tech, there are a couple of projects out there with java and msmq, but i'd rather not have to add lots of dependencies in the project

What other types of queues are available that might easily plug into traccar without having to load any/many dependencies ?

any advice or pointer appreciated

alternatively, i could use addDynamicHandlers in BasePipelineFactory
But I cannot find any documentation or forums posts to get me going on this section

[tananaev]

I think we should use one of the standard messaging protocol. For example, AMQP or MQTT. Latter might be a better option because it is designed for telemetry data like GPS location and sensor data. Many other GPS tracking and IoT systems support it.

[quade1664]

good point Anton, I totally forgot about MQTT.
It isn't a queue, but it knows if the receiving end is offline and simply stores the messages internally, and then sends when its online, this will be sufficient, i'm unsure about persistent storage (server restart etc)

I guess there is nothing baked into traccar that uses MQTT so would need another dependency? if you know of any I should look at please let me know.

I'm a bit confused on the best section to implement this
For example, this article:
https://www.traccar.org/forums/topic/forward-gps-data-to-mirror-server/
suggests using dynamic handlers

Would I simply add this to config file and create a new class with that name to handle the data to send?

[tananaev]

I guess we need another dependency. Make sure you do investigation on available solutions to ensure that it's well maintained popular library.

I think you can implement it similar to existing forwarding handler.

[jpmens]

I'd very much like to see MQTT there, and I believe the Paho Java MQTT implementation is rather solid. We've had excellent experience with MQTT (C, Python, etc.) and greatly appreciate the fact that it's very lightweight and that there are several Open Source brokers to choose from (Mosquitto, VerneMQ and others).

It might amuse you to know that we actually currently forward to a program which then publishes to MQTT :-)

[ckrey]

I think Eclipse Paho is the choice https://github.com/eclipse/paho.mqtt.java

And, as MQTT is supported by most Queueing systems as an input channel (RabbitMQ, etc.), it can be used with a variety of systems, not only pure MQTT brokers.

I would like to support the implementation, but my experience is in C, Python, not Java. But I stand by for testing.

Another thought: if we implement MQTT to forward Positions, we should should do likewise for Events.

[quade1664]

Have a look at the last link on this page
#1582

Does it look like this has already been done?

[jpmens]

Judging by the README, yes, but this looks quite like a way I would not implement it, as re-assembling all the bits on an MQTT subscriber side would be a lot of work.

/traccar/device/ivan/latlon -99.999999,-99.999999
/traccar/device/ivan/fixtime 2018-03-25T23:59:23Z
/traccar/device/ivan/speed 0.0
/traccar/device/ivan/protocol osmand
...

I'd be much more in favor of having the JSON payloads we currently have in HTTP when a position and an event is POSTed.

[quade1664]

Thanks, so you think each piece of data is being posted via mqtt instead of one payload?
That certain sounds like a pita!

I've got it up and running and it looks like its eventually sending everything in one go, but I could be wrong as this is new area for me

This is the line that looks like is does the sending:
https://github.com/ivanfmartinez/ivanfm-traccar-mqtt/blob/master/src/main/java/com/ivanfm/traccar/mqtt/ProcessPosition.java#L198

However I get an error

org.apache.velocity.exception.ResourceNotFoundException: Unable to find resource 'mqtt-alarm\unknown.vm'

From the googling i've done, it looks like an intelliJ environment issue.
I'm in double digits in years on visual studio and single digits in days on intelliJ so i'm stumped!

[tananaev]

The error has nothing to do with intelliJ. Looks like you are just missing some templates.

[quade1664]

Thanks Anton
@jpmens do you have any ideas on how to fix this, or pointers where to look
this is beyond me now, and seems like the final hurdle

[jpmens]

@quade1664 the example I showed above, taken from the fork's README indicates they are publishing each value individually; no doubt about that. See also: ivanfmartinez/ivanfm-traccar-mqtt#2

I cannot help you with Java stuff whatsoever, but I think velocityContext has something to do with Templates (I believe that's what Traccar uses to format, say, SMS and emails on notifications. In other words, it would appear as though just that file is missing, and it should contain some tokens (I suppose) which will be expanded by that invocation.

I see something like velocityContext.put("geofence", geofence); which I assume adds a variable called geofence to the template's environment. Judging by the little I know from traccar, that means the template file will have to contain a $geofence which is then expanded to that value.

Guesswork. :-)

[jpmens]

Create a file and see what happens:

$ mkdir mqtt-alarm
$ echo Hello > mqtt-alarm/unknown.vm

[ombasaMukhwami]

I found RabbitMQ to be the best solution. Added few lines of code from their site and it is working well

[quade1664]

Thanks @jpmens, i'll check it out

[quade1664]

Hi Anton
I need to publish my changes to the staging server for testing
I would like to publish in release mode, and not debug mode

Are you aware of any documentation or support forums where this might have been covered previously?

In intelliJ i've included all dependencies and built the artifacts, but its created 42mb file - which looks too big
Should I be replacing JAR file: /program files/traccar/tracker-server.jar
That file is only 1.2mb

I assume i'm on the wrong track here
Is there anything special I need to do for upgrading existing installation?

[tananaev]

I usually just package JAR file using maven directly, not from IDE.

[ombasaMukhwami]

@tananaev can you put the dependancies for AMQP and MQTT so that we can just replace the tracker-server.jar without the error of
com/rabbitmq/client/ConnectionFactory - NoClassDefFoundError (WebDataHandler:155 < BaseDataHandler:29 < ... < ExtendedObjectDecoder:70 < ...)
Let me send via mail

[quade1664]

I think you'd better put these dependencies in yourself

I assume you have tried publishing with maven as anton suggested, but its not been referenced
Anton, does he have to edit a file somewhere to specifically include this dependency in the release?

[tananaev]

We do have a forwarding queue now and support retries.