New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrating Mbed LoRaWAN Stack in Mbed-OS 5.8 #6087

Merged
merged 40 commits into from Feb 16, 2018

Conversation

Projects
None yet
8 participants
@hasnainvirk
Contributor

hasnainvirk commented Feb 13, 2018

Description

Reviewed in feature-branch

Status

READY

Migrations

| NO

hasnainvirk and others added some commits Nov 27, 2017

Adding base class for all LoRa radio drivers
All Mbed-OS drivers for LoRa radio devices must implement
this pure virtual class in order to be compliant with Mbed-OS
applications.

This class comes loaded with all necessary data structures.
The implementations of this class can come out of tree.
Adding base class for LoRaWAN interfaces
All network interfaces for LoRaWAN protocol must implement this
class. In order to be compatible with Mbed-OS applications, any
implementation of this class must use the data structures and
Mbed-OS timers provided.

lorawan_data_structures may look repetitive but this is essential
as we have a plan to use a reference implementation for LoRaWAN mac
layer from Semtech. Some of the data structures provide seemless
transition from semtech implementation (as MAC layer) to the Mbed-OS
control layers above.

features/lorawan/lorastack is the placeholder for future items like mac and
phy layers. system/ will contain all the common bits.
Adding PHY layer for LoRaWAN
LoRaPHY is the abstract class for the LoRa PHY layer which governs
the LoRaRadio and provides some common functionality to all regional
implementations.
We support 10 regions and every region comes loaded with default parameters.
These parameters can be changed by the Mac layer or explicitely by the stack
controller layer using APIs provided. This layer in essence detaches Mac completely
from PHY and provides more modular approach to the entire system.
Apart from class structure, the internal functionality is directly deduced from
semtech reference implementation that's why most of the internal data structures are
used on 'as is' basis.
In addition to that, the PHY layer provides APIs to control the LoRaRadio layer, i.e.,
the lora radio driver, ensuring that the radio is accessed from a single entry point.
A seperate data structure file is added which is common to PHY layers only.
Adding MAC layer for LoRaWAN implementation
The actual mac algorithms are being used as it is in the reference
implementation.

We introduce an internal class that starts a thread and constructs an event queue
to handle deffered calls from interrupt context for RTOS. The code base is
compatible with Mbed-OS 2 as well.

GetPhyEventHandlers() API provides mac callback funtions for PHY layer,
which are in turn delegated to radio driver from the PHY layer.

LoRaMacInitialization() is augmented with LoRaPHY parameter which let's
the MAC layer know which particular PHY layer is in use.

LoRaMacSetTxTimer() and LoRaMacStopTxTimer() are used when duty cycle is
off for testing purpose or to support custom application timers.

If the duty cycle is off, mac and phy layer work togather to figure
out the next possible transmission time.

LoRaMacCrypto APIs are provided which provide seemless integration of
mbedTLS into mac layer for cryptography. User application is supposed to
provide proper mbedTLS configuration file.

All other APIs are retained as it is.
Adding LoRaWANStack class to control MAC and PHY
LoRaWANStack class is our controller layer on top of our
current MAC and PHY layer. It provides services to an implementation
of LoRaWANBase class.

It is a singleton class owing to the fact that the mac layer underneath
is not a class object. Instead, it uses the MAC via setting mib, mlme, mcps
requests and getting responses back from the mac layer using confirmations and
indications.

In essense this class is a special handle for
mac layer underneath which is predominantly reference design based.
In future we may refactor the LoRaMac.cpp code to make it object oriented
and cleaner.

At one end, it binds the application selected radio driver with the PHY layer
and at the other end it provides services to upper layers handling the mac via
well defined APIs.

For proper selection of a PHY layer, user must use Mbed config system.
For this purpose an mbed_lib.json is provided which can be overriden by the
user defined mbed_app.json. By default the EU868 band is selected as a PHY layer.
User must set relevant keys for the selected connection mechanism.
Adding LoRaWANInterface - implementing, LoRaWANBase
This class is the doorway for the user application into the
Mbed-OS implementation of LoRaWAN protocol. It implements LoRaWANBase
and hence would work with any stack implementation underneath, ensuring
seemless portability for applications.

It takes a pre-constructed object of LoRaRadio and delegates it in the
downward direction. Before calling connect() user must call initialize() function
in order to initialize stack and mac layers.

connect() APIs can be used to either provide relevent keys and connection method at
runtime or compile time (using Mbed config system).

enable_adaptive_datarate() and disable_adaptive_datarate() are used to turn on/off
automatic rate control. Otherwisem set_datarate() could be used to set a particular
data rate on the current channel.

set_confirmed_msg_retries() is valid only for CONFIRMED messages. It means that the stack will
retry for a given number of times before timing out.

set_channel_plan() and get_channel_plan() are used to set or get a particular channel plan.
These APIs are particularly useful in case of ABP (activation by personalization). Because
in case of OTAA(over the air activation), by default the stack takes in a CF List (carrier frequency list)
sent by the base station in conjunction with Network server. This list overwrites all user configured
channels or channel plan. set_channel_plan() can be used to set a single channel as well by setting the
parameter for number of channels to 1.

remove_channel_plan() or remove_channel() are used to remove a currently active channel plan or a specific
channel.

send() and receive() APIs follow posix design except the socket descriptor is replaced with port number here.

lora_event_callback() API is used to set a callback function from the application side which is used by the stack
to inform user of particular events like CONNECTED, DISCONNECTED, CRYPTO_FAILURE, TX_TIMEOUT etc.
[IOTCELL-279] Using Application provided EventQueue
The EventQueue thread in LoRaMac.cpp is disbanded and the LoRaWAN
protocol is redesigned to store a pointer for an application
provided EventQueue. It means that now the stack runs in the
same thread as application. Application provided EventQueue is used
to defer ISRs from radio driver and timer callbacks as well as the
application events are queued to the same event loop.
Receive API typo fix
Receive API should return the length of data written to the user buffer
as the Posix APIs suggest rather than sending the pending length of data
back.
That has actually been a typo mistake which actually wnt in even with doicumentation :)
[IOTCELL-270] Detaching TxNextPacketTimer
TxNextPacketTimer callback was being used for testing only (compliance testing to be precise).
Now there are independent methods and direct calls to automatic timers for the
compliance testing so there is no particular need for this timer anymore.
Use EventQueue for elapsed time
Current implementation uses high resolution timers to calculate elapsed time.
This prevents for example deep sleep completely and causes unnecessary timer
events.

This commit changes implamentation to use EventQueue::tick() to get elapsed time.
Reworking callback API
Application should be able to add some optional callbacks if it needs to.
Ofcourse there is a penalty of 8-12 bytes per callback, but there can be
certain needs of the application that needs to be met for example setting
up a link check request etc.

We have introduced a structure that contains callbacks for the application use.

 - 'events' callback is mandatory, user must assign it. Because this callback brings
   state changes for the application. We cannot segregate this into individual handlers
   because of RAM penalty.
- Other calbacks (none of them are implemented yet are optional).

Example of using the API is provided with doxygen
Move all compliance test code inside compilation flag
All compliance test related codes are now inside LORAWAN_COMPLIANCE_TEST
build flag. This will reduce memory usage in when compliance test codes
are not needed.
Making static objects SingletonPtr
We had a lot of static objects which would get constructed and hence
pull in some of the LoRaWAN code into the builds for other technologies.
Such objects have been now lazily initialized using utility class
SingletonPtr.
Bug fix in setting data rate
Setting up user defined data rate was found broken maybe because of
some rebase issue. Code has been setting always the default data rate
and ignoring used defined values.
Architecture rework, bug fixing & missing features
MAC layer is now a class rather than being a blob.
In addition to that Mac commands are now being handled in
a seperate subsystem (a class of its own). In future we
will do the same with othe sublayers of MAC like MLME, MCPS etc.

The drive behind this exercise is to make MAC and supporting layers
into an object oriented system.

Major bug fixes include:

- last join time inclusion in band parameters
- disabling rx2 window if we missed the slot already
- MLME uplink schdule hook
- nbRep according to spec
- maintaining datarate after successful joining
- suppressing MLME requests if MAC is in TX_DELAYED state
- Uplink dwell time verification

Some missing features are implemented. Details are as follows.

Support for LinkCheckRequet:

An application API is added, add_link_check_request() to delegate a
request for Link Check Request MAC command.

* Application provides a callback function that needs to be called on reception of
  link check response.

* Mac command is piggybacked with data frames.

This API makes the sticky MAC command stick with the application payloads until/unless
the application un-sticks the said mac command using remove_link_check_request() API.

Handling fPending bit:

If in the Downlink, we get the fPending bit set in fctrl octet,
we attempt to send an empty message back to Network Server to
open additional Receive windows. This operation is independent
of the application. An RX_DONE event is queued bedore generating
the said empty message. Specification does not mention what can be the
type of that empty message. We have decided it to be of CONFIRMED
type as it gives us an added benefit of retries if the corresponding
RX slots are missed.

Radio event callbacks as Mbed callbacks:

radio_events_t structure has been carrying C-style callbacks which was
inherited from the legacy code. These callbacks has now been changed to
Mbed Callbacks that makes sure that we can be object oriented from now
on.
Changing Timer callback to Mbed Callback
Time handler class had a c style callback attached to it which
had been hampering us to be fully object oriented.
That particular callback is changed to Mbed Callback which is attatched
to a specific object hence allowing us to be fully object oriented.
Change LoRaWANTimer to a C++ class
LoRaWANTimer is now called as LoRaWANTimeHandler class as this class handles both
current time and timer functionalities.

Some refactoring on how LoRa objects are created was needed:
- LoRaWANTimeHandler object is created by LoRaWANStack and shares with LoRaMac and PHY.
- LoRaPHY object is now member of LoRaWANStack class instead of static variable in source file.
Timer SingletonPtr & mac protocol data structure
Ticker objects embeded in TimerEvent_t data structure were getting constructed
even for the non LORAWAN builds. And that's what was bloating the builds.
We now lazy initialize them using Mbed-OS utility clas SingletonPtr.

A central data structure has been created that carries all the protocol level
variables for the Mac layer. This is important as we are going to break down
MAC services into subsystems and we will pass around common data using that data structure.
[IOTCELL-289] Adding Mlme class handle
MAC layer will services will be broken down into independent
subsystems. This is the first of those efforts. We have introduced
LoRaMacMlme class that handles everything related to MLME subsystem or
subservice. To accomodate subsystems we have grouped all protocol level
variables into one big data structure. A pointer to that data structure
will be passed around the subsystems in order to regulate the entire
system.

LoRaMac::Send() and LoRaMac::SetTxContWave*() APIs are made public as
they are needed to be accessed by the subsystems.
[IOTCELL-290] Adding MCPS subsystem
Like MLME, MCPS has also been alloted its own class. This is the
2nd stage of breaking down the MAC services into subsystems.
Changing MLME data access methods to inline
As indicated in one of the reveiws, it makes more sense
to change the data access methods to inline as they are
just one liners.
[IOTCELL-288] Adding MIB subsystem
As a part of MAC layer breakdown into independent subsystems, we have
introduced a class for MIB service.

Major changes from the baseline are:

- making OpenRxWindow() public in LoRaMac.cpp
- Moving various data structures to central protocol data structure
Remove duplicate configuration flags
This commit changes code to use directly mbed os configuration system generated
compilation flags.
[IOTCELL-286] Adding Channel Plan subsystem
Channel planning was distributed over LoRaWANStack and LoRaMac previously.
We now centralize it by allocating the service to its own class.

Thus making the workflow consistent, i.e.,

Request for channel plan = Application->Interface->stack->Mac->ChannelPlan

Major change apart from adding the channel plan subsystem are the API changes
in LoRaMac class.
[IOTCELL-282] Code cleanup/simplification and rules
Baseline is changed to use a single set of data structures that simplifies the
code in the LoRaWANStack and Mac layer. We are now following certian rules for naming
data structures.

- All structures visible outside their domain are prefixed as 'lorawan_'
- All mac structures are prefixed as 'loramac_'
- All subsystem or module strucutures carry their name in prefix, like 'mcps_'

PHY layer still have legacy camel case data structures which will be entertained
later while we will be simplifying PHY layer.
Test cases are also updated with the new data structure naming conventions.

One major difference from the previous baseline is the removal of static buffer
from mcps indication. And we do not copy data from stack buffer to rx_msg buffer.
This saves at least 512 bytes.

It may look like now that if we have received something but the user have not read
from the buffer, then the buffer will be overwritten and we will lose previous frame.
Yes, we will. But the same will happen even if we would have copied the buffer into rx_msg
because then the rx_msg gets overwritten. So we decide to abandon copying the buffer at
multiple locations. We inform the user about reception, if the user doesn't read and
the data gets overwritten, then so be it.
[IOTCELL-346] Removing default value storage
We now save roughly 500 bytes by removing storage of default
parameters in the loramac_params_t data structure. We use Mib to
get default values from PHY whenever needed instead.

loramac_sys_arams_t now contains only the runtime values set during operation
whenever defaults are needed we directly query the PHY layer or via Mib as the
need maybe.
Major PHY layer modifications
The PHY layer had a lot of duplicated code in various geographic regions.
In this commit we have tried to concentrate all common functionaliy into
one single class which LoRaPHY that provides three kind of methods:

i) Non virtual base methods which are there for upper layer use, e.g.,
   providing access to driver or generic PHY layer functionality which
   needs to be exposed to upper layers.

ii) Virtual methods (no hard limit on implementation) that can be overriden
    in derived classes. Some PHY implementations will need that as they may
    come with very peculiar channel schemes, e.g., dynamic channel schemes
    in US bands.

iii) Protected methods which are only available for the derived PHYs

We have adopted a mechanism for the dervied PHYs to announce their differenmtiating
parameters in their constructors by filling up a data structure known as lora_phy_params_t
which exists at base level. Access modifier for this data structure is protected so it can only be
used by the base or derived classes, i.e., no exposure to upper layers.

For extra functionality and differentiating controls, a derived PHY can override any virual method as necessary.

In addition to that we have adopted the Mbed-OS style guide and have changed data structures and code to reflect that.
Some data structures are removed.

* Algorithm to get alternate DR is modified. Current scheme, works as multiples of 6 as EU and EU like PHYs
provide 6 datarates. We make sure that we try a datarate at least once. If nuber of join retries is a multiple
of 6, we may try multiple times on each data rate.

* Most of the PHYs with dynamic channel plans, always override the above mentioned algorithm as the rules governing
  this algorithm do not hild in their case.

* band_t data structure is enhanced with lower band frequency and higher band frequency. That enables us to validate
  frequency based upon the band and hence we can have a single function for all PHYs to validate frequency.

* In some PHYs, there were some extra channel masks were defined which were not being used. Hence removed.

* EIRP table corrected in some PHYs based upon spec.

* PHY functions in response to Mac commands are renamed to reflect what they exactly do.
  for example accept_rx_param_setup_req() because that's what they do. they can either accept
  the mac command or reject it.# Please enter the commit message for your changes.
Use EventQueue for timers
Instead of initiating own timer objects we can use EventQueue::call_in() method
as we already have handle to EventQueue object.

Also setting timeout and starting timer has been combined to TimerStart method.
@hasnainvirk

This comment has been minimized.

Contributor

hasnainvirk commented Feb 13, 2018

@AnttiKauppila @kjbracey-arm @sg- @cmonr @0xc0170 PR made to land on master.

@hasnainvirk

This comment has been minimized.

Contributor

hasnainvirk commented Feb 15, 2018

@0xc0170 We need CI here. I am not sure why ci-morph-test status is not reported yet even after two days.

@0xc0170

This comment has been minimized.

Member

0xc0170 commented Feb 15, 2018

/morph build

@0xc0170 0xc0170 requested review from kjbracey-arm and AnttiKauppila Feb 15, 2018

@0xc0170

This comment has been minimized.

Member

0xc0170 commented Feb 15, 2018

@AnotherButler New API

@mbed-ci

This comment has been minimized.

mbed-ci commented Feb 15, 2018

Build : SUCCESS

Build number : 1148
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/6087/

Triggering tests

/morph test
/morph uvisor-test
/morph export-build

@0xc0170 0xc0170 requested a review from SenRamakri Feb 15, 2018

@kjbracey-arm

Can you link to the original review PRs in the description?

No further code comments - I said everything on those.

I assume that root-level license file and other license issues have been cleared with someone? Don't recall seeing any discussion.

@mbed-ci

This comment has been minimized.

@0xc0170

This comment has been minimized.

Member

0xc0170 commented Feb 15, 2018

/morph build

@mbed-ci

This comment has been minimized.

mbed-ci commented Feb 15, 2018

Build : SUCCESS

Build number : 1150
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/6087/

Triggering tests

/morph test
/morph uvisor-test
/morph export-build

@mbed-ci

This comment has been minimized.

@mbed-ci

This comment has been minimized.

@mbed-ci

This comment has been minimized.

@cmonr

This comment has been minimized.

Contributor

cmonr commented Feb 15, 2018

I assume that root-level license file and other license issues have been cleared with someone? Don't recall seeing any discussion.

Holding off on merge until @kjbracey-arm's license question is addressed, and to give @SenRamakri and @AnttiKauppila a chance to review.

@AnotherButler

This comment has been minimized.

Contributor

AnotherButler commented Feb 15, 2018

How many of the .h files will get their own user or contributing API reference pages? Also, would you like copy editing of the .h files?

@hasnainvirk

This comment has been minimized.

Contributor

hasnainvirk commented Feb 16, 2018

@cmonr It is good to go now.

@cmonr

This comment has been minimized.

Contributor

cmonr commented Feb 16, 2018

@kjbracey-arm BSD-3 is fine to use. There are a couple of instances of it being used within our repo, and partner software. There also appear to be a couple of mentions in the handbook, but it could probably be stated more explicitly that BSD-3 is ok to use.

@hasnainvirk I'm going to merge this in, but would you still mind linking the original review PRs in the description like @kjbracey-arm requested?

@cmonr cmonr merged commit 1e1a178 into master Feb 16, 2018

18 checks passed

AWS-CI uVisor Build & Test Success
Details
ci-morph-build build completed
Details
ci-morph-exporter build completed
Details
ci-morph-test test completed
Details
continuous-integration/jenkins/pr-head This commit looks good
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
mbed-ci-generic Build finished.
Details
travis-ci/docs/ Local docs testing has passed
Details
travis-ci/events/ Local events testing has passed
Details
travis-ci/littlefs/ Local littlefs testing has passed
Details
travis-ci/mbed2-ATMEL/ Local mbed2-ATMEL testing has passed
Details
travis-ci/mbed2-MAXIM/ Local mbed2-MAXIM testing has passed
Details
travis-ci/mbed2-NORDIC/ Local mbed2-NORDIC testing has passed
Details
travis-ci/mbed2-NUVOTON/ Local mbed2-NUVOTON testing has passed
Details
travis-ci/mbed2-NXP/ Local mbed2-NXP testing has passed
Details
travis-ci/mbed2-SILICON_LABS/ Local mbed2-SILICON_LABS testing has passed
Details
travis-ci/mbed2-STM/ Local mbed2-STM testing has passed
Details

@cmonr cmonr removed the needs: review label Feb 16, 2018

@0xc0170 0xc0170 deleted the feature-lorawan branch Jun 27, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment