Thingstream Integration

[jamal-mouline]

Thingstream has currently couple SDK solutions, including one that runs on a Nordic nrf52840, using the nRF5.

Looking to port that Thingstream SDK on the same target hardware but using nRF Connect instead.

ubxlib is already very well integrated with nRF Connect, so the idea is to extend it to support Thingstream.

Thingstream current design leaves it to the "integrating" application to handle modem power up / power down.
But requires the port of a "serial transport" to handle sending and receiving data stream.

To accomplish this, first thought is to pursue the following architecture approach,
where the "integrating" application would fully rely on the existing ubxlib / uDeviceOpen / uDeviceClose to manage modem power up / power down, create a UART instance, etc...
Once the modem is powered up, and the UART interface established, tap into the device handle obtained from the uDeviceOpen, in order to access uPortUartRead and uPortUartWrite, (and maybe others) from
https://github.com/u-blox/ubxlib/blob/master/port/platform/zephyr/src/u_port_uart.c
to implement the "serial transport" for Thingstream

An obvious challenge is to make sure that when there is ongoing “thingstream” operations, the UART interface would be exclusively dedicated to it. I am not sure whether the current usage of a gMutex lock/unlock was designed to provide just that?

Please let me know whether this is a sound approach to pursue with...
Otherwise, definitely open to better alternative approaches and suggestions...

Thank you

===

First reply copy/paste here from email trail...

We are aware of the Thingstream SDK but no-one on the ubxlib side has experience of actively using it, though we do know the right people to contact; we can involve them as necessary.
Whether you can use the uDeviceOpen()/uDeviceClose() APIs for this or not will depend upon whether the Thingstream SDK needs to react to URCs emitted by the cellular module: the uDeviceOpen()/uDeviceClose() APIs will start a ubxlib AT client on the UART, which will be the thing reacting to URCs from the module and the two will likely clash. If the Thingstream SDK does not need to react to URCs then this won't matter; if it does then you will need to drop down to lower-level ubxlib APIs instead and things will be trickier.
Regarding protection against simultaneity, I suspect you will have to do that at application level as ubxlib will not be aware of anyone else using the same transport as it. Basically, if you're doing Thingstream things, don't call ubxlib APIs.
When you open your issue, can you clarify if your aim, at application level, is to have Thingstream stuff that just happens to use the ubxlib porting layer for convenience, or do you actually want to use the ubxlib APIs as well as doing Thingstream stuff? The former is trivially easy, the latter will be complex.

[jamal-mouline]

Re:
Can you clarify if your aim, at application level, is to have Thingstream stuff that just happens to use the ubxlib porting layer for convenience, or do you actually want to use the ubxlib APIs as well as doing Thingstream stuff?

Answer:
At this time I would go with the former assumption. But I will double check and confirm later...

@RobMeades
Confirmation:
To support the Thingstream transport layer, I only need to use the ubxlib porting layer from uPortuart.c / uPortUartRead/Write.
So far, there was no need to use any ubxlib APIs

[jamal-mouline]

@RobMeades
Put together a quick PoC
"
Once the modem is powered up, and the UART interface established, tap into the device handle obtained from the uDeviceOpen, in order to access uPortUartRead and uPortUartWrite in support of the Thingstream serial transport
"

Turns out the read && write required the uart instance handle (instead of device handle!)

I was able to retrieve the uart handle in two fashions from the two pointers passed to uDeviceOpen: deviceCfg and deviceHandle
they both equate to 1 in my test case --code snippet below
is this a coincidence or always going to be the case?
if not, I think the one obtained through the deviceHandle is more reliable but wanted to double check?
did I miss a more direct API that retrieves the uart instance handle to use for uPortUartRead /Write?
Could not find one?

Thank you

int32_t errorCode = uDeviceOpen(&pObj->deviceCfg, &pObj->deviceHandle);
if (errorCode == 0)
{
uDeviceCellContext_t *pContext = U_DEVICE_INSTANCE(pObj->deviceHandle)->pContext;
int32_t uartHandle = pContext->uart;

  LOG_INF("uartHandle %d deviceCfgUart %d", uartHandle, pObj->deviceCfg.transportCfg.cfgUart.uart);

[RobMeades]

Hi there: getting the UART handle from the device configuration is fine but you should avoid calling U_DEVICE_INSTANCE() as that is not an API (i.e. it is not in an api directory), it is internal ubxlib source code and may be changed without notice.

EDIT: sorry, I misunderstood, the UART handle is not in the device configuration, pObj->deviceCfg.transportCfg.cfgUart.uart is the UART number (i.e. if you're using UART 0 then it will be 0, if your using UART 1 then it will be 1, etc.). This might be the same as the handle, or it might not; it entirely depends upon the implementation of the porting layer, which you should not rely on.

To get the UART handle through function calls you would call uCellAtClientHandleGet() on the device handle to get the AT client handle and then call uAtClientStreamGetExt() on that to get the UART handle, something like this:

uAtClientHandle_t atHandle;
uAtClientStreamHandle_t stream = U_AT_CLIENT_STREAM_HANDLE_DEFAULTS;
int32_t uartHandle = -1;

// Assuming that deviceHandle contains a valid device handle, returned by uDeviceOpen()
atHandle = uCellAtClientHandleGet(deviceHandle);
if (atHandle != NULL) {
    uAtClientStreamGetExt(atHandle, &stream);
    if (stream.type == U_AT_CLIENT_STREAM_TYPE_UART) {
        uartHandle = stream.handle.int32;
    }
}
if (uartHandle >= 0) {
    // You now have the UART handle
}

EDIT: but if you only want to use the Thingstream SDK with the ubxlib porting layer, you only need the ubxlib porting layer, why waste code/RAM on anything else: just open the UART and use it (i.e. uPortInit() then uPortUartOpen())? For that matter, why not integrate with Zephyr's UART? Less code in the way, which is always good.

[jamal-mouline]

Re: pObj->deviceCfg.transportCfg.cfgUart.uart
Understood, not the proper source to get the uart handle.
I am using UART1 indeed to communicate with the SARA on the Sparkfun.

To get the uart handle, the combination of using uCellAtClientHandleGet and uAtClientStreamGetExt per your suggestion works well.

Good point about your second EDIT. Will keep that in mind. But for the current integration/evaluation, I think we want to keep them both. For example, it is likely that we re-use the ubxlib cell configuration, http client, and others...

Thank you.