Decoupling "inventory" from LoRa Server (looking for feedback)

[brocaar]

I'm thinking about decoupling the "inventory" part from LoRa Server. I think this makes the whole setup more flexible, as it allows you to use a different data storage, data schema etc, different transports between the lora-app-server and the application receiving (and sending) payloads. Below a draft diagram:

This means that the Application / Node and Channel(List) APIs will be moved to the new lora-app-server, and that loraserver only has knowledge about a node-session.

My idea is to keep the RESTful and gRPC api with (optional) JWT auth in lora-app-server, all other gRPC interfaces will be (optional) client side certificate auth only as they should not be exposed to the outer world directly. Of course this means there will be some breaking changes, but I think apart from moving the API from one service to the other, we can limit the impact.

Looking forward to feedback and thoughts.

[ghost]

Yes, it may be a good idea to move (optionally) authorization credits away from loraserver. From my point of view, we can use Radius or LDAP backend as "lora-app-server" and let redis cache credits. Radius is robust, but not for mio/s requests.

[VirTERM]

That is actually very good idea

On Aug 20, 2016, at 12:10, xPrcek notifications@github.com wrote:

Yes, it may be a good idea to move (optionally) authorization credits away from loraserver. From my point of view, we can use Radius or LDAP backend as "lora-app-server" and let redis cache credits. Radius is robust, but not for mio/s requests.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[ivajloip]

Talking about radius, this might be interesting for some people. @arun1587 (one of the authors of this draft) created a prototype implementation (I am not sure if he will be able to share it publicly though).

[arun1587]

https://github.com/arun1587/radius/tree/lora-radius --> i have the prototype of the draft here (the server side).

[brocaar]

Yes, with the decoupling, that will all be possible :-) I think for the sake of simplicity, I will provide a lora-app-server which has the same "inventory" functionality as the current LoRa Server (application / node / channels) and will use PostgreSQL. That way Radius will not be a requirement to get started with LoRa Server + it is easy to upgrade to the new structure (as the db table structure will be the same). However, I'll also provide the gRPC stubs, so it will be really simple to implement your own custom app server based on the same api.

[brocaar]

All the otaa / uplink and downlink tests are working again after a big refactor (see https://github.com/brocaar/loraserver/tree/app_server_refactor/internal/testsuite for the testcases). There is still a lot to do, as the rest is still failing and I haven't started yet on lora-app-server.

You can find the first definition of the apis for each component here (look at the .proto files): https://github.com/brocaar/loraserver/tree/app_server_refactor/api

There will be some changes in the queueing of data down payloads. After the refactor this will be handled by the application-server. (for class A) LoRa Server will ask the app server for a downlink payload and provides the app server with the max payload length. See also https://github.com/brocaar/loraserver/blob/app_server_refactor/api/as/as.proto#L81

[siscia]

I am a little concerned about the use of gRPC, especially for communicating with the lora-app-server that is suppose to maintain the inventory (why could simply use GET request), handles join request (simple POST) and think about crypt (here gRPC sound more appropriate, however a little lonely).

Similarly between the lora-controller and the loraserver, why do we use a more standard REST interface?

In my opinion, REST would be more standard and simpler to use for most programmers. Also it allow a loosely coupling between the component, with gRPC you have to define all the interface upfront.

Maybe I am missing something but I don't see the benefit of gRPC (performace?).

Just a thought, the leadership of @brocaar in this project is amazing and the product really well done and I will trust him with whatever he choose to be more appropriate.

[brocaar]

Hi @siscia thanks for your feedback! There are a couple of reasons I think gRPC is a good choice:

I wouldn't say REST interfaces are more standardised, but they are more accessible / open. E.g. the transport is just plain text instead of a binary protocol so it is easier to debug. My problem with REST interfaces is that you need a proper framework or else you have to be really good at documentation writing. When an API becomes more complex, it is really easy to make a mistake (either you forget to document fields, or as an user, you forget to implement them, make wrong assumptions about data types).

The advantage I see in using gRPC is that the API is defined in a .proto file which is really easy to interpret and you don't have to have any code in place when specifying out the API definition. Of course I could write it out in a text file, but the great thing about the .proto file is that I can generate API clients and server stubs for many different languages. This means that when you're planning to let your project interact with one of the LoRa Server components, you can generate your own API clients or server stubs (or use the provided Go client / server stubs) and you'll be 100% sure that you have a correct interface. As well (depending your programming language), your compiler will warn you about mistakes (e.g. missing methods).

The lora-app-server component will have a REST interface (most of the loraserver API will be moved to the lora-app-server), and guess what, it is generated based on the same .proto files ;-) (using https://github.com/grpc-ecosystem/grpc-gateway).

[brocaar]

Just a quick update on the progress (from the outside it might seem that this issue takes ages to complete)! The loraserver refactor is as good as complete (there might be some really small API changes). I'm now working on the last lora-app-server backend bits. When that has been done, I'm planning to migrate the AngularJS code to ReactJS (the AngularJS code was hacked quickly together and I think it can be done better) and update the documentation, as there are quite some new steps involved to setup a LoRa Server architecture.

[brocaar]

And LoRa Server 0.12.0 and LoRa App Server 0.1.0 are out :-) Please see the changelogs for upgrade information + https://docs.loraserver.io/lora-app-server/ for LoRa App Server documentation.

Looking forward to your feedback (and bugreports 😁)!

[vikkymahdrey]

hi brocaar,
I'm receiving packet on lora server from our configured gateway with ble node and receiving packet can view with mosquitto_sub -t "gateway//rx but packet not storing into the postgresql .
I checked it in frame_log but table is empty their.
Is it will not save packet automatically?
Please help us on this. THanks