DAM/MDM9206 SDK Example and compile instructions

[Thalhammer]

Are there instructions about how to compile and link DAM applications for SIM7000 without having to buy the multi-thousand dollar ARM IDE?

Compiler GCC or Clang?
Simple makefile for demo code?

[botletics]

Everything should be here.

[Thalhammer]

Yes I saw that folder and also played around with it, but I can't get it to compile using default clang.
The compiler distributed by Qualcomm is no option since you need to have a company to get a copy (I don't have one).
And the Arm compiler is also no real choice, since you can only use the trial for 30 days and I can't spend that much money for a single project on a compiler I'll never need for anything else.
How do you compile those examples ?

// EDIT:
I just installed the trial version of those arm tools and followed the readme of this repo:
https://github.com/SIMComHub/sim7000c_DAM_demo
Now it just hangs trying to connect to this license server, which it obviously can't.

[botletics]

Hmmm not sure, sorry. I honestly haven't tried it yet. I've been super busy with other stuff like adding FTP functions and tutorial.

Oh, by the way, SIMCom told me "regarding sim7000E module, we don't suggest you to develop c code and run inside module". I guess that means don't use DAM on SIM7000E. Not exactly sure why lol.

[Thalhammer]

Being unable to run my code directly inside the module really was the biggest reason why I choose Simcom modules in the first place.
If this does not work it would be really bad.

[botletics]

A SIM7000A user told me that apparently there are some SDK examples here. Here's what he said works:

1. Install SIMCOM’s drivers for Windows. After this, when I connect the SIM7000 Shield’s USB to my PC, it shows SIM7000’s ports in Device Manager.
2. Then run the SIM7000_EATAPP_DT.exe in the SDK’s DAM_DEMO folder. Set the DIAG Port and AT Port to the COM# shown in the Device Manager. Then press the START button. After a few seconds, it will download dam_demo program to the SIM7000 module.
3. Solder two wires to Pin49 (UART3_RXD) and Pin50 (UART3_TXD). The DAM_DEMO uses UART3 for user interface.
4. I am using a SiliconLabs CP210x USB to TTL convertor. I connected the UART_RXD from the SIM7000 to TXD of the CP210x, and UART_TXD from the SIM7000 to RXD of the CP210x. (The CP210x shows up as COM6 in Device Manager in my PC).
5. Open a Serial Terminal program (I am using Putty). Set the COM port to the COM Port of the CP210x, and set the Baud Rate to 115200. Open a session.
6. In the Putty, send 1 as command. This should then display the Main Menu for the DAM_DEMO program. Now I can go into each option and try out the functions.

[Thalhammer]

Thanks, this will definitely help me in the future, but for now, I am nowhere close to having something I could download to the module. I need to compile the example code and if possible I'd like to use a free compiler for it (not the ARM one). Qualcomm seems to provide a llvm based one but I could not find any place to download it.

I'll get a trial license for the arm compiler and I might even get the initial implementation done within the trial period, but later bug fixing will be quite a challenge if I need to get a new trial license anytime I need to fix a bug.

// EDIT: In case I get everything working, would you merge a PR containing samples and build files for this kind of use case?
I'll create some samples to try things out anyway, so it would be no problem to share them here.

//EDIT2: Using a trial license from arm compiling worked fine (at least I think so since there were no errors. I can't really test it since my modules are still in postage :( )
But I would still want to get llvm working since the license will only be valid for 30 days.

[Thalhammer]

So I finally have my modules and could play around with them a little bit.
I compiled the example in this repo and uploaded it to

• /custapp/cust_app.bin (Alternate File System)
• /customer/cust_app.bin (Alternate File System)
• /custapp/cust_app.bin (Primary File System)

None of which resulted in any output on UART3.
I know that UART3 works since I redirected NMEA to uart3 to test and it worked fine.
The techship link does not help since one needs a company to download it and I have no company.
Do you have a compiled app you could post here so I can make sure my upload procedure works correctly?
Could you maybe get me in touch with the SIM7000A user that apparently got it to work ?

EDIT: Nevermind, I got it working using the demo from SimcomHub.

[bujtasadam]

Hi, I can not able download the SDK from here: https://techship.com/news/software-development-kit-now-available-for-simcom-sim7000-series-lte-cat-m-nb-iot -GSM-cellular-modules /
I registerd but the link seems ded to me.
Some one can upload it for me?
ty your help.

[blazczak]

Well done @Thalhammer getting this to work, your experience with it seems rather painful - my initial steps in getting started with this have been similar, in that Qualcomm isn't really maker-friendly.

Do you have a gist or a write-up posted somewhere of all the (working) downloads necessary to get set up promptly? Since the license is so short-lived, it would be great to be able to get set up quickly.
Also, the reason for not choosing GNU Arm Embedded Toolchain over the ARM compiler: is it because they support completely different chips thus no way to compile this stuff with the Toolchain?

[Thalhammer]

@blazczak Hi,
You might wanna look here:
https://github.com/Thalhammer/simcom_dam
It has a bunch of examples for the api, various utility code, a rewrite of the vendor libs for gcc and a bunch of information about internals of the module as well as guides for common errors and how to debrick the module if you screwed up.

Even with this though its far from User friendly.

[blazczak]

Thanks, I got that link from Timothy already. The repo has code but no supporting software I don't think. Do you have a list of prerequisite downloads to get started? It would be great for folks starting out, like me.

[tschloeter]

For which platform do I need the gcc cross compiler to get that working?
Thanks in advance.

[Thalhammer]

@blazczak @tschloeter You need to get the arm cross compiler, as the SIM7000 uses a cortex-a7.
I tested both gcc-arm-none-eabi-9-2019-q4-major and gcc-arm-none-eabi-7-2018-q2-update, but I am pretty sure a newer one will also work.
You can download them here.

[tschloeter]

Great, thanks for your reply. I will try

[Thalhammer]

@tschloeter
I don't know what you intend to do with the module, but if its going to be a commercial product, I highly recommend reevaluating that hardware choice. I also updated the readme of my repo to reflect his.
The repo is not really production ready and effectively unmaintained.
The module firmware is buggy, undocumented (except for some headers) and hard to debug.

[tschloeter]

It's a project for my personal purposes with no commercial background.

What I am hoping to do with DAM is creating a lightweight interface protocol between SIM7070G and my microcontroller. Talking AT protocol and having to configure all MQTT registers using AT cleartext ASCII commands takes a lot of resources on the microcontroller. If possible I would like to offload most of the MQTT logic into the SIMCom Module itself and only send values to the chip.

Also I don't like the AT protocol as it does not provide a data length on URCs for example. If you receive a response from an MQTT subscription you never know if you are still reading the response or the next URC. So, it would be easy to inject code into an MQTT response.

If I could parse the MQTT responses directly in C code on the SIMcom module and only return values I am interested in, data security would be much higher.

As I have not yet finished my HW design: Do you have a suggestion of a better-suited hardware module that can do LTE-M, GNSS and MQTTS?
By the way, I am also having a hard time to get the TLS part of MQTTS working.

EDIT:
Btw, I also wrote a mail to SIMCom years ago and never got a response. They don't respond to hobbyists I guess. And I am also embarrassed by not having publicly available datasheets and software documentation. But I have not yet found an alternative. Anyone tried Quectel modules?

[Thalhammer]

The Quectel modules use a very similar chip (In fact some of my sources is based on derived quectel sdks, so they are certainly an alternative). Mqtts is one of the problems I mean. It checks the certificate time against the current time of the builtin rtc. However the api to set a time for this rtc is either broken or not implemented (its not in any header and if you try to directly call the api it just crashes the module), so that time will always be 1970 + module uptime. So you either need a cert thats valid for this timeframe or you need to disable certificate validation (which effectively breaks ssl) or compile in a custom build of ssl+mqtt which takes its time from a different source (like ntp or gps).

[tschloeter]

Ok, and have you already tried one from Quectel whether they have a better software quality?

[Thalhammer]

Unfortunately not, however there is a platform.io framework for them and if I remember correctly they definitly had a way better documentation. I don't know if they have better firmware though.
@Wiz-IO can probably tell you more about it.

[blazczak]

@Thalhammer @tschloeter In that case I guess I should count myself as "lucky" since I've received all of two responses from them. 😃 Each of the responses was anonymous and without any signature or mention of title within the company: "please tell us where is your company.and more about your company information" and "hi Sir We do not support SIM7100 use embedded application now,if you Just for personal interest thanks". So basically my experience with that company has been essentially equal to yours. heh

I'm sitting on a few different modules (7000G, 7100A, 7600A-H) on various PCBs, and so far my experience has been painful as well. I was actually planning to purchase more of the 7100s but I am now waiting to see how usable they are first.

Because the AT interface is so unusable my primary goal behind wanting to compile my own firmware for 7100X has been to implement a simple app running on the module that would serve as an encrypted communication tunnel to an HTTP(S) or MQTT(S) server forwarding incoming bytes on its RX and returning bytes from the server on its TX with minimal initial configuration (client_id, keys, certificates on module; stuff which is already configurable through the AT interface or the QPST tool); essentially replacing the cumbersome AT interface on one of the UART/USB ports. This way you can skip all of that AT silliness and just send/receive bytes from/on your microcontroller/SBC, and re-use this firmware for various projects requiring a connection with a server.

In normal circumstances this should be easy to do. Before you completely switch context to other projects @Thalhammer, based on what you've seen with their API so far, how easy do you think this would be to implement given the hurdles and bugs you had uncovered?

As a side note, I'm wondering if with all of these issues you're describing these modules are even able to come close to the advertised speeds (50Mbps uplink for SIM7100X, for example) over the software interface alone, before network speed variations.

If all of the above fails, I'd be happy to take recommendations on what brand+module or brand to purchase to get a better and more reliable out-of-the-box LTE connectivity experience.

[tschloeter]

I have currently ordered some Quectel BG96 modules to evaluate them. I could share my experiences here if you are interested. They are going to arrive in CW13.

[Thalhammer]

@blazczak

encrypted communication tunnel to an HTTP(S) or MQTT(S) server forwarding incoming bytes on its RX and returning bytes from the server on its TX with minimal initial configuration

Yeah should be doable, SSL might be a bit problematic though due to the issues I outlined above. You might have to disable certificate checks or check them yourself.

able to come close to the advertised speeds (50Mbps uplink for SIM7100X, for example) over the software interface alone

It's not that the api would be slow or that the module would not perform as advertised. In fact if it works, it works really good. The problem is getting it to work in the first place which can be really fragile. E.g. in the early days I had an issue where I printed a string to console and the length of the string resulted in the module not booting or working fine (it turned out a section had to be a multiple of 4 in size and depending on the string size it was or was not). Stuff like that is easily fixed once you know it, but it took me for ages to figure out, simple because theres no usable documentation about this stuff.

That said I highly doubt you'd need 50Mbit/s since you'd need to use usb to push this bitrate to the chip and there are better options if you already have usb.

@tschloeter I'd be interested in this as well.

[datamann]

I have currently ordered some Quectel BG96 modules to evaluate them. I could share my experiences here if you are interested. They are going to arrive in CW13.

I would love to hear what you think of them :-)

[blazczak]

I have currently ordered some Quectel BG96 modules to evaluate them. I could share my experiences here if you are interested. They are going to arrive in CW13.

With 375Kbps down and 375Kbps up it's too narrow-band for my needs, but it will def be good to hear your folks' experiences with other manufacturers' modules.

[blazczak]

Yeah should be doable, SSL might be a bit problematic though due to the issues I outlined above. You might have to disable certificate checks or check them yourself.

Sounds good, I wonder if I could use the mobile network time or an RPi-based RTC to seed it. Last resort I'll just do an end-to-end encryption over a non-TLS connection.

The problem is getting it to work in the first place which can be really fragile. E.g. in the early days I had an issue where I printed a string to console and the length of the string resulted in the module not booting or working fine (it turned out a section had to be a multiple of 4 in size and depending on the string size it was or was not). Stuff like that is easily fixed once you know it, but it took me for ages to figure out, simple because theres no usable documentation about this stuff.

Thanks for the info, the work you've put in reverse-engineering the firmware and time spent maintaining your repo @Thalhammer With little to no support from simcom and the firmware API resembling a minefield due to the number of bugs/issues, such documentation work is invaluable to your fellow makers.
I see last check-ins from ~10 months ago - before you abandon it, can you think of any remaining source code or notes from your research that you think would be good to check-in for the future? I'll start implementing my proxy app soon, based on your repository.

[tschloeter]

With 375Kbps down and 375Kbps up it's too narrow-band for my needs, but it will def be good to hear your folks' experiences with other manufacturers' modules.

In my case not an issue as my application's main processor is an ATSAMC21 with some kBytes of RAM. I wouldn't know where to go with all that bandwidth and decided to go for an LTE Cat M device to take advantage of its (theoretically) better network coverage.

Anyway I think Quectel is using quite similar controllers throughout their portfolio, so you should be able to find a comparable broadband chip. However I found that for hobbyists the Quectel modules seem harder to solder as they are using BGA connections in most modules.

Indeed you can find layout-compatible modules for 2G, 3G and LTE so you can just swap the chip in your existing design and reuse your PCB which is really great to try out different technologies.

[Thalhammer]

Sounds good, I wonder if I could use the mobile network time or an RPi-based RTC to seed it. Last resort I'll just do an end-to-end encryption over a non-TLS connection.

The issue is not that you cant get an accurate time. The issue is you cant supply it to the ssl implementation provided by the firmware. Since all the memory is writable, one could probably patch this, however you would have to reverse the firmware to do so. If you compile your own ssl stack (like libressl) and your own mqtt library (the builtin can not use ssl callbacks) it's not an issue as well, as you can just supply your own function to get the time.

There are actually multiple sources to get the time:

• GPS very accurate and usuable without Network, takes some time to get a fix and might not get one at all, but if you need the location anyway you get UTC Time for free.
• Network EDGE and other networks broadcast time in some conditions, however it depends heavily on your network provider. I've seen the message about it once, but never got the network to do it reliable.
• NTP Network time protocol, if you have to connect to the network anyway you can implement an ntp client. if you skip all logic to compensate network rtt and jitter a clean implementation is about ~400 lines (you wont get < 1s accuracy but thats seldom needed anyway)
• Builtin AT RTC The AT interface contains RTC commands to set the internal rtc, but for some reason the changes are not visible in the dam, you can however run AT commands from the dam to read/write it.
• Byproduct Some protocols might give you the time for free without any effort (e.g. most http servers send the "Date" header which contains the current time)

I did an application using this chip for my bachelors degree (complete with custom pcb and both server/client code).
Basically it connected to an mqtt server, sent its location (if available), some sensor data and battery voltage to server and went back to sleep.
For this I also included the timestamp (even though in retrospect I would not have needed to), I used unencrypted mqtt to make it easier.
To get the time I had logic similar to this:

startGPS();
startNetwork();
waitForGPSFix(); // variable but I used 1 minute, if you dont have a fix after a minute you likely wont get one at all.
if(hasGPS()) {
    useGPSTimeAndUpdateRTC();
} else if(hasRTCTime()) {
    useRTCTime();
} else {
    doNTPRequest();
    useNTPTimeAndUpdateRTC();
}

This worked pretty well for my use case. You might want to switch RTC and NTP cause I have no idea how stable the RTC is.
I wanted to save network traffic, cause my mobile card was billed per traffic, so I prefered the rtc once its set correctly.
You might also want to outright just do NTP cause its pretty reliable and accurate enough and by far the fastest if traffic is not an issue for you. If you dont connect to the network anyway, then just connecting for NTP is probably a bad idea.

can you think of any remaining source code or notes from your research that you think would be good to check-in for the future?

I believe theres some code scattered around my pcs which is not yet in the repo, but I am not sure anymore what of it worked and what did not.

go for an LTE Cat M device to take advantage of its (theoretically) better network coverage.

I'd highly recommend double checking this. Our network claimed the same and at some point it might actually be true, however in reality at the time I started there was not a single Cat-M network available in my area and about 10 months ago EDGE coverage was still way better than both Cat-M and Nb-Iot. So even if you have a Cat-M network at your place, I'd highly recommend a module that supports EDGE as well if you want broader/global coverage. Note that the sim card needs to support this as well.

[tschloeter]

So even if you have a Cat-M network at your place, I'd highly recommend a module that supports EDGE as well if you want broader/global coverage. Note that the sim card needs to support this as well.

True. Not knowing a lot of LTE-M and NB1, I bought an eval board for an NB1/Cat M/2G board and wondered that I could only get access to 2G with my SIM (it was Telefonica / O2 Germany).
With Telekom Germany I was able to get LTE-M working but also not in the basement garage as I was hoping.

Cat NB1 seems to be reserved for B2B at least in Germany. I could not manage to get a SIM from any provider to test it because they strictly sell to business partners. Anyway the data rate of NB1 could be sufficient for my needs but the data plans would be much too expensive if you do not earn money with the data.

Anyway I think as with every new technology there is always a tradeoff between chance and risk. I think I will use a 2G / LTE-M capable module and see if the LTE-M coverage will get better over time. Vodafone Germany have started their first steps in LTE-M three weeks ago as I have read.

[tschloeter]

I have today received my Quectel BG96 modules and as promised let me tell you my first results:

MQTTS works out of the box. Configured, uploaded CA cert, connected.

Also I have tried out their native API (called QuecOpen). Using PlatformIO as an IDE I was able to execute my first Hello World app within 2 hours. They have a community containing very useful information. Most of the APIs are available by kindly asking or some of them on GitHub. Multiple firmware branches providing different APIs are available. Tons of useful tools for flashing, developing and maintenance are available through the community. I will give their modules a try in my projects.