The project is tested on Windows 10.
- Two nRF9160 Thing Plus boards.
- J-Link, for flashing application to the board and debug.
- Two UART-USB adapters.
- 串口调试助手 (Serial Debug Assistant). Other serial port tools on PC are possible, but this tutorial uses the aforementioned tool.
- HxD, a hex editor.
- Audacity, an audio process software.
- nRF Connect for VS Code
Install the nRF Connect tools by following the guide. Import this project by the nRF Connect VS Code extension. Consult their guide for importing project.
(In case you need to select the board for creating new application, the board is circuitdojo_feather_nrf9160_ns.)
In main.c file, comment out one of TX_BOARD and RX_BOARD defines to make the board either a transmitter or a receiver (but of course, one board need to be TX and another one to be RX). Build and flash.
Each pin name printed on the nRF9160 board corresponds to a pin number, listed in the following figure. Pin number is used in the wiring tables.
| Board | UART-USB adapter | |
|---|---|---|
| UART2 | TX: P0.24 | RX |
| RX: P0.23 | TX |
| Board 1 | Board 2 | |
|---|---|---|
| UART1 | TX: P0.00 | RX: P0.01 |
| RX: P0.01 | TX: P0.00 |
The TX board will receive the original audio from PC, and use Codec 2 to encode the audio, then transmit the Codec 2 bits to the RX board.
The RX board will receive the Codec 2 bits from TX board, and decode the bits, then transmit the reconstructed audio to PC.
We use hts1a.raw as input audio. It is in 16-bit PCM format with 8000 Hz sampling rate.
Each board has two serial ports connected to PC:
- Type-C port: One for connection of the USB Type-C port on the board to the PC. This port is used by
printkand log function for console output. - UART2 port: Another one for the UART2 port connected to a UART-USB adapter, and finally connected to PC. For TX board, this one will receive the original audio from PC; for RX board, it sends the processed audio to PC.
For TX Board:
- Type-C port
- Open Serial Debug Assistant, choose the correct
Serial Port, setBaud Rateto115200. - Click
Open serail port. - Push the reset button on the board, then some information will be printed out.
- Open Serial Debug Assistant, choose the correct
- UART2 port
- Open Serial Debug Assistant, choose the correct
Serial Port, setBaud Rateto115200. - Click
Open serail port. - Check
Send a file, then there is a window prompted to choose the file.
- Open Serial Debug Assistant, choose the correct
TX Board Type-C Port
For RX Board:
- Type-C port: same as TX Board.
- UART2 port:
- Open Serial Debug Assistant, choose the correct
Serial Port, setBaud Rateto115200. - Check
HEX display. - Click
Open serail port.
- Open Serial Debug Assistant, choose the correct
RX Board Type-C Port
In the Serial Debug Assistant window for TX board UART2 port, click the right lower "paper plane" icon, to start sending the original audio.
TX Board UART2 Port
After TX board received the whole audio, it encodes the audio and send Codec 2 bits to the RX board. After RX board receives all the Codec 2 bits, it decode and send the reconstructed audio to the PC by UART2 port.
Therefore, in the Serial Debug Assistant window for RX board UART2 port, many hex strings will appear. After the transmission stop, click Save data. There will be a window prompted to choose the file location. The data is saved as plain text (txt) file.
RX Board UART2 Port
Open HxD, create a new file (Ctrl+N), copy all the content in the txt file to HxD, save as a .raw file.
HxD and txt
Open Audacity, click File->Import->Raw Data..., choose the .raw file just saved. Change the sample rate to 8000. Then we can play the audio.
The nRF SDK currently DO NOT support UART 16-bit data transmission, functions in the name like uart_xx_u16 won't work. Therefore, although the audio buffer is uint16_t, the data between transmission should use uint8_t type. So, always pay attention to the data size.
When using the Kconfig of the nRF connect extension for VS code, use Save to file to permanently save the configuration in the prj.conf. If you use Save, it only saves to a temporary file and will be lost after build, and the change doesn't take effect.
In Details View, you can find the file circuitdojo_feather_nrf9160_common.dts, which contains all the peripheral settings.
When we want to change, for example, UART1, we need to write the configuration into an overlay file. If we didn't create it before, there will be a sign No overlay files. After click it, an empty overlay file will be generated.
At the beginning I tried to include Codec 2 files to the application as a library, but ended with Conflicting CPU architectures error. I then tried to export the environment of zephyr and set those for Codec 2 library, but ended with argument to '-O' should be a non-negative integer, 'g', 's' or 'fast' error. However, it was very strange that as I saw from the command line the option with -O is valid.
Anyway, before I gave up, I tried to include Codec 2 as application source files, then it successfully compiled and linked.
Zephyr Device Tree: I think the document is good to learn the device tree (with helpful knwoledge to configure the peripherals in overlay file), long documentation but most parts are understandable.
Zephyr Example Projects: There is only description on each example page, but there is also a link to the GitHub repo so you can read the code.
Zephyr API Documentation: Documentation of functions in Zephyr. The asynchronous UART part is especially helpful for this project.
Error Numbers: Good to know for debugging.
To make the project build, I also commented out code about Codec 2 700 bps mode. It is possible to include that code, but for quickly make the whole system work, I chose to not bother that. Therefore, the supported Codec 2 rate is from 1200 bps to 3200 bps.
Other important changes are that where there is if(n)def ARM_MATH_CM4 or if(n)def CORTEX_M4, I added ARM_MATH_CM33 and CORTEX_M33. Especially, I added the code in codec2_fft.h:
#ifdef ARM_MATH_CM33
#include "nrf9160.h"
#include "core_cm33.h"
#include "arm_math.h"
#include "arm_const_structs.h"
#endifCurrently, the application transmitter role saves the whole input audio, whose duration is 3s, in memory. Then it encodes and also saves the Codec 2 bits in memory. Next, the transmitter sends the Codec 2 bits to the receiver. The receiver also save the whole encoded audio bits in memory, and then decode.
For semi-live or live audio (streaming), the above pattern is not feasible. We need to encode the audio once a frame (320 16-bit PCM data for 1200 bps, 160 16-bit PCM data for 3200 bps) comes in. Then transmit the Codec 2 bits while receiving the next frame.
The 8000 Hz sample rate means that 320 PCM data corresponds to 40 ms duration, and 160 PCM data corresponds to 20 ms. Moreover, these are the durations we can use to encode the frame. The UART can be set fast enough so that the transmission time can be omitted.
If in 1200 bps mode, the encoding time is longer than 40 ms, then semi-live or live stream is not possible. This then requires a faster clock frequency of the CPU.