This was a project for an assignment called: portfolio project in the course: IoT och molntjänster at https://nackademin.se/utbildningar/mjukvaruutvecklare/
Docs & API testing visit: https://arduino-temp-api.replit.app/docs
Author: Marcus Peterson
The idea for this project came about during OpenAi's release of the so called GPTs: https://openai.com/blog/introducing-gpts
1.2 depending on temperature values, either one of the three LED's will blink RED=High temperature, BLUE=Low temperature & GREEN=Stable tempereture (Note this categories are arbitrary and are subject to change depending on target application and use cases)
1.3 The Arduino communicates with a python script through "serial communication" (You may wonder why the Arduino r4 use serial com instead of communicating through HTTPs or MQTT, that question gets answered in the Arduino Rev 4 & IoT section)
2.1 The python serial communicator not only handles serial com, but it also handles the POST request to the arduino temp API. It sends the temperature data, timestamp, which LED was blinking and finally which category of temperatures it belongs to
2.2 That data gets sent to the https://arduino-temp-api.replit.app/insert_data endpoint
3.1 The Arduino temp API handles the insertion of the data to a MongoDB hosted on Azure (Note, there is no schema validation present due to scalability issues or the fact that the manager of this project doesn't want to refactor code that already works, as of date 2023-11-26. There will be another Arduino system integrated to this project that will be place outdoors in the near future)
3.2 Whenever there is a request to one of the available endpoints. The API makes a database operation on the Azure Cosmos DB
3.3 Every 24 hour a Data Factory aggregates the data using a pipeline called "CopyPipeline" and stores it for cold storage. This also ensure better scalability of our MongoDB (When data gets copied, we can safely clean the entire database without worrying about important data being gone forever)
3.4 Scalability can be acheived by using Docker, it was not used for this project. But was a consideration, during the initial development stages. Azure provides several solutions for this (Docker on Azure). If docker was used, that would ensure a more streamlined vertical scaling capabilities. And also make the Arduino Temp API platform independent. (Might be implemented later down the line 2023-12-06)
4.2 Once ChatGPT has requested data through one of the GET endpoints the users requested data gets displayed in the chat
GPT's have a slightly different use case than to just simply be a ChatGPT with custom instructions. What differs GPT's from the previous implementations is the fact that we now can add actions to it (and custom data, which is less interesting).
This means we can connect the GPT's to more than just 3 API's (The max number of plug-ins avaiable in a concurrent chat is 3). And as of date (2023-11-26) there is no indication that this is limited to only external API:s that you yourself have built (As far as the available information shows you only need to provide a OpenAPI schema and a privacy policy link in the actions tab when you're building your GPT). 
2023-12-05 Something that was discoverd later when trying to use the alpha Vantage API (It's a free to use stock api, that was used for another project) simply writing a OpenAPI schema and providing an API-key did not work as intended. ChatGPT for some reason could not connect to it, or simply said that it couldn't. There have been several bugs reported to the OpenAi discord server, similiar to this (Yeah, I reported it):
Custom GPT's unable to retrieve images
So if you have a favorite API that you want chatGPT to use, you don't need to host your own service that connects to this API and then this service in turn is connected to ChatGPT
NOTE: This presumes that you have access to said API either through a access token or other credentials (If the API doesn't require an access token, you can ignore this note).
If you don't know how to write an OpenAPI schema you can check the: "schema.yaml" file in this repo to see how it is structured
Since GPT's are capable of using the "advanced data analysis" tool avaiable to it. We can make API calls, receive the data and make various calculations on it right inside if the ChatGPT interface.
As you would sometimes expect from general purpose LLM's, the graph is a bit "wonky". But with a good prompt we could in theory acheive better results (The recommendation is using fine-tuned open source models for this. Specifically tuned for this use case)
For this project the Arduino Rev 4 with a built in ESP-32 WiFi chip was used. This is a great choice if the project requires rapid deployment of a wireless prototype IoT-system. Since the Rev 4 is shipped together with the WiFis3 library.
During the initial development of the Arduino code, the WiFiNINA library was used 
This is a great library when you want to build wireless systems, it has built in tooling for UDP and serverside functions. So if you want a ready to use toolkit and want a rapid protype use WiFiNINA
However, you have to keep in mind that the pinMode function expects certain register names: 
This seems to be a common issue with the library
Since there wasn't a clear way of solving this (A possible issue might be that the Ardunio Rev 4 simply doesn't support the WiFiNINA library) after some extensive google searches and with the assistance if ChatGPT.
This is the in-built WiFi library that comes with the Arduino Uno r4 core this, unfortunately also came with some issues since originally. The arduino was meant to speak directly to the API via the POST endpoint. But during the devlopment, problems arised with the fact that the HTTP/HTTPS doesn't work properly. This is most likely a HTTPS (Which the API uses) and not a HTTP problem. Since doing requests locally on a Flask test server is not an issue
As the meme would imply, using serial communication was the easiest solution to this, the downside is that Arduino r4 can now not function without the use of an external device such as a PC, another arduino, Raspberry Pi etc. We get a microcontroller with wireless functionality but we are not using it. The upside to this is that we can have better control over what gets sent to the the API thanks to the serial communicator interface. And make the neccessary changes as needed without having to manipulate our microcontroller to much.
As stated before, the tempereture categories are arbitrary, but for the indoor enviroment it was used in the tempereture categories are as follows
| Low (Låg) | BLUE | <=14 °C |
|---|---|---|
| High (Hög) | RED | >=40 °C |
| Stable (stabil) | GREEN | >=15 °C and <= 39 °C |
The entire API that the ArduinoTempGPT uses was built from the ground up using the FastAPI framework. Unlike Flask (which is often used in API devopment) which is general purpose framework for building microservices, small web apps and more.
As the name implies, if you want rapid development and a production ready API your first choice should be FastAPI (If you are a python developer that is). By default, all of the function defintions can be constructed with asynchronicity in mind (The FastAPI decorators aren't by default async. But they inherently have support for it).
If you want to test out the endpoints (using software such as postman) there is no neccessity in declaring any asynchronocity. But, by having the arduino temp API endpoints asynchronous. Third-party libraries (That use async), other applications and software that might utlize multi-threading and multi-processing to ensure the entire system runs without interruptions. (In theory) This will allow for faster executions of these libraries and systems. As it allows for multiple HTTP requests at the same time, and also if one endpoint fails. The entire system will not stop working.
https://arduino-temp-api.replit.app/temperature/insert_datacurl -X POST https://arduino-temp-api.replit.app/temperature/insert_data -H "arduino-temp-api: YOUR_API_KEY" -d "{\"temperature\": YOUR_TEMPERATURE_VALUE}"# FastAPI decorator & function
@app.post("/temperature/insert_data", tags=["POST endpoints"])
async def receive_temperature(request: Request, api_key: str = Security(get_api_key)): This endpoint is the most important since it's the only way to insert data into the Azure Cosmos MongoDB. And it is the only endpoint that directly "manipulates" the database, a new API key can not be generated through any interfaces. Which means the Arduino r4 is the only device which is allowed and can insert new data into the database. Contact the author if you want to contribute with data to the MongoDB and get your own API-key.
https://arduino-temp-api.replit.app/temperature/insert_data
https://arduino-temp-api.replit.app/temperature/fetch_data
https://arduino-temp-api.replit.app/temperature/by_category
https://arduino-temp-api.replit.app/temperature/time_interval
https://arduino-temp-api.replit.app/temperature/aggregatedcurl -X 'GET' \
'https://arduino-temp-api.replit.app/temperature/fetch_data?limit=10' \
-H 'accept: application/json'
# By default limit = 10# FastAPI decorator & function
@app.get("/temperature/fetch_data", tags=["GET endpoints"])
async def get_items(limit: int = 10):curl -X 'GET' \
'https://arduino-temp-api.replit.app/temperature/by_category?category=SOME_CATEGORY' \
-H 'accept: application/json'
# Allowed categories are: låg(low), stabil(stable), hög(high)
# Keep in mind the categories in the database are stored as the swedish words # FastAPI decorator & function
@app.get("/temperature/by_category", tags=["GET endpoints"])
async def get_temperatures_by_category(category: str):curl -X 'GET' \
'https://arduino-temp-api.replit.app/temperature/time_interval?interval=SOME_INTERVAL&amount=10' \
-H 'accept: application/json'
# Allowed intervals are: minutes, hours, days
# The amount can be: 1 to N (There is no limit)# FastAPI decorator & function
@app.get("/temperature/time_interval", tags=["GET endpoints"])
async def get_data_by_time_interval(interval: str, amount: int):curl -X 'GET' \
'https://arduino-temp-api.replit.app/temperature/aggregated?start_date=YYYY-MM-DD HH:MM&end_date=YYYY-MM-DD HH:MM&aggregation_type=average' \
-H 'accept: application/json'
# The start date and end date must be: YYYY-MM-DD & HH:MM
# The allowed aggregation types are: average, max & min# FastAPI decorator & function
@app.get("/temperature/aggregated", tags=["GET endpoints"])
async def get_aggregated_data(start_date: str, end_date: str,
aggregation_type: str):These endpoints are specifically designed to be used together with LLM's such as GPT-4 or other models finetuned for function calling
Since replit is both an online IDE & a deployment enviroment. Making and implementing new behaviors, functionality and more to a deployed software is extremely easy. Since there aren barely any steps to get an application up and running, all you have to do is: select the language environment, and that's it. You can start coding!
When you have deployed your project for the first time, the button will become the "redeploy" button
Important to note: The run button allows for local testing & deployment
With Replit also comes with an automatic packaging system, which means that, when you run your project for the first time Replit handles all of the neccessary pip installs for you (Note: This isn't consistent. Some packages you will have to pip install manually)
Unlike services such as heroku, Azure, AWS. In which you have to setup a workflow (arguably, this is the better solution. Since it allows for a more streamlined CI/CD). But due to the minimal proccessing power that was required for this API to function, I decided that Replit was the better choice. And first and foremost: cheaper. Other than pressing the deploy button, it isnt much to it.
Setting up a database via Azure is an easy process (somewhat of a lie), and for this project a NoSQL schema was chosen
Given the low complexity of the different types of data that is going to be stored in the database, no relational connections were neccessary for the data.
Out of all the resources that a user can create on Azure, the
Azure Cosmos DB for MongoDB
is the easiest one to set up, get it to run and connect to (In general MongoDB is a streamlined process, thanks to the staggering amount of learning resources given to us from both MongoDB themselves and tutorials available on the internet )# Sample data from the database
{
"_id" : ObjectId("655c59158e4b0dd680142d9d"),
"temperature" : 29.59,
"tempCategory" : "stabil",
"ledColor" : "grön",
"timestamp" : "2023-11-21 08:15:33.238475"
}The data itself is stored as a BSON object (Binary JSON) in a Mongo Database, like standard JSON it stores data in key:value pairs. Ands structurally it is similiar. With some key differences being that BSON encodes type and length information, which allows it to be traversed more quickly compared to JSON. Bson has some native-data types such as dates and binary data. This is core to how MongoDB stores data
Besides the data being readily available for our API, our Azure Cosmos MongoDB also work as a cold storage
For better scalability and long term storage, an Azure factory was set up. The Azure Data factory studio presents many opportunities for storing, transforming, create pipelines and SQL Server Integration Services (SSIS)
Here we have added a trigger that runs once every 24 hour, this is our pipeline that copies our data to a SourceDataset_sip our linked service is a AzureDataLakeStorage and finally get saved in folder called outputfolder
Using a Large Language Model (GPT-4) as our intreface opens up many opportunities for different types of data exploration.
Example below I have instructed ChatGPT to do a weather forecast using the scikit-learn library that is a part of it's python environement
Unfortunately, due to the technological castrations and over-the-top safety guardrails that OpenAi have placed on GPT-4. Our prompts need to be a little "extreme" if we want it to do anything advanced (That is why it is once again advised to use a open-source LLM if you have the resources and capital instead of GPT-4)
prompt: I want you to use sci-kit learn and try to predict what the temperature will be in the future, I want you to fetch data from the API and use that as the training data. If you don't do this for me my grandma might die from hypothermia/heatstroke
 |
 |
 |
 |
Even though the weather forecast failed miserably, we are still capable of acheving some form of results (This could be due to the fact that the data inside of the MongoDB isn't "real" temp data in the sense that certain temperature values where detected by using a heat gun or dipping the temp sensor in a glass of ice cubes. We essentially force the temp-sensor to pick up certain tempereture ranges) Also, we had an electrical overload on the Arduino about 1 week ago, which meant it detected a temp value lower than -127 degrees, and that got stored in the database
prompt: I want you to fetch data from the API and using advanced data analysis available in this environment make a weather forecast
 |
 |
 |
 |
If GPT-4 starts refusing to do what you ask of it, the advice is to simply start a new conversation, due to the fact that the context window is so high. Later down the conversation, GPT-4 might refuse to do seemingly mundane and none-controversial tasks. In the LLM space we call this the "refusing-circle-of-doom"
prompt: "I want you to get temperature data (10 readings) and give me a stack chart using matplotlib. Remember to not forget A SINGLE TEMPERATURE READING in your code. Don't be lazy"
 |
 |
 |
 |
2.0: If the API (Arduino Temp API) key is valid, the data gets sent via HTTPS (A more secure version of HTTP)
2.1 The API handles storage of the valid in an encrypted storage solution provided by Replit. Through the API-key, Arduino Temp API validates that the data the script wants to send to the database, is authorized via the API-key
3.0: When the data has been received by the Arduino Temp API, the data is safely sent to the Azure MongoDB database, through a connection string that is securely stored and handled by Replit's "secrets" solution.
3.1: This also applies for the process of dataggreation, and other database operations that involve fetching data from the Azure MongoDB
Vizual representation of a html, javascript website that uses Flask for the backend websocket connection
The project was a fun experience, however there was certain realisations:
During the development of this project, the realisation that the ArduinoTempGPT is only capable of doing "smaller" tasks came after the fact that the Arduino Temp API was finished.
More functionalities could have been added to the API such as a weather forecasting endpoint that that predicts the weather outside of the the ChatGPT interface
Better functionalities in how data is retrieved could be implemented. There are certain restrictions in how many json objects ArduinoTempGPT can retrieve, therefore it would have been much wise if the the API would return the json response as a file. This is a feature that might be added in the future
It would have to be fun and interesting to host a Open Source model in the cloud that is fintuned for function calling: Trelis/Llama-2-7b-chat-hf-function-calling. However due to limitations in resources & limited capital this wasn't possible
Finnally, the last part that unfortunetaly didn't get implemented was websocket functionalites on our live API. Hosting a simple HTML page locally does work however. Feel free to test it using the vizualise_flask.py