This is an Arduino example of including the size and crc of a file in a stream of data, and then using the expected size and CRC at the end to verify the data was validly transmitted. This is done with the help of the CRCStream class included in the example. The main benefit of the CRCStream class is that the size and CRC can be determined as the stream is read in (and possible used). While you do have to read the entire stream to verify, you are not required to hold the contents in some in-memory array just to perform the CRC check. It is handled as the stream is read.
This is an Arduino sketch that uses an Adafruit AirLift (or compatible ESP32 board) to connect to a WiFi network, monitor a TCP port for data, and when the data is available process it to verify that the data was transmitted with no errors. For an example, it is rather verbose, with code to connect to the network, open a TCP port, read data from the TCP port, etc. I wrote this code as part of project to perform over-the-air updates of Teensy firmware. And it will be integrated into that example. But I felt that retaining a separate example could be useful.
The code uses a very straight forward implementation of the crc32 algorithm
from Björn Samuelsson.The values generated by this code should match the values generated by the linux tool
crc32, which you will be required to install.
sudo apt-get install libarchive-zip-perl
Please note that this CRC mechanism is only used to verify that the data was validly transmitted. It does not provide any security or
privacy for the data transmitted.
The CRCStream class wraps a given stream, and is also given an expected size and CRC. As the stream contents are read, CRCStream keeps track of the number of bytes read and their overall CRC value. Once the stream use is complete, the program can call the sizeAndCRCMatch to verify that the stream contents match the expected size and CRC. If they don't match, then something went wrong with the transmission of the contents.
The GetBytesTask monitiors a TCP port for data. When data is detected,
the code will read from the beginning of the data to get the expected size and CRC of the data that follows. The size and CRC are expected to
be prepended to the beginning of the file as text separated by the '!' character.
<expected size>!<expected crc>!<the rest of the data...>
Once the expected size and CRC are pulled from the data stream, GetBytesTask processes the rest of the stream contents using an
instance of CRCStream. Once all the data is read, it then calls sizeAndCRCMatch to verify it matches.
This is a bash script that will take a file, prepend the original file size and CRC to the original file, and then send the modified file to a specified IP address and port using TCP. You can use this to transmit a file to the Arduino to be processed by GetBytesTask. The script uses the stat and crc32 command line tools to get the original file size and CRC, and the netcat command line tool to transmit the modified file over TCP.
In order to connect to WiFi, the WiFi SSID and password are stored in secrets.h. Make sure you don't check it a modifed version your own code.
As mentioned, this code is written to work with an Adafruit AirLift board. The AirLift requires a specific fork and version of the WiFiNINA library, so make sure it is installed in your Arduino environment.
This example also uses two of my own libraries, Arduino Logging and Task Manager. These libraries are only required for the example, not for the CRCStream class itself. You can comment out or change the lines as desired.