Update to latest firmware

README.md

@@ -1,9 +1,15 @@
 # Friend: Open-Source AI Wearable with 24h+ on single charge
 
-|                                                             Assembled                                                             |                                                             Disassembled                                                             |
-| :-------------------------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------------------------------------: |
-| <img src="https://github.com/BasedHardware/Friend/assets/883804/59a67cc8-e337-4e86-824b-638718619e77" width="450">                | <img src="https://github.com/BasedHardware/Friend/assets/883804/e524c4f9-de2c-47a5-b634-9fdd2772b345" width="450">                   |
-
+<table>
+  <tr>
+    <td align="center"><img src="https://github.com/BasedHardware/Friend/assets/883804/59a67cc8-e337-4e86-824b-638718619e77" alt="Assembled" width="450"></td>
+    <td align="center"><img src="https://github.com/BasedHardware/Friend/assets/883804/e524c4f9-de2c-47a5-b634-9fdd2772b345" alt="Disassembled" width="450"></td>
+  </tr>
+  <tr>
+    <td align="center">Assembled</td>
+    <td align="center">Disassembled</td>
+  </tr>
+</table>
 
 [![Discord Follow](https://dcbadge.vercel.app/api/server/kEXXsnb5b3?style=flat)](https://discord.gg/kEXXsnb5b3) &ensp;
 [![License: GPLv3](https://img.shields.io/badge/license-GPLv3-blue)](https://opensource.org/license/agpl-v3)
@@ -67,7 +73,7 @@ There are 2 different apps in these repositories located in different branches a
 
 ## Getting Started
 
-Follow these steps to get started with your Friend. Note, we tested everything on a mac + iphone. Currently trying to make it work for Android
+Follow these steps to get started with your Friend.
 
 ### Install the app
 
@@ -80,7 +86,7 @@ Follow these steps to get started with your Friend. Note, we tested everything o
 
    - For AppWithWearable, open file api_calls.dart located in `AppWithWearable/lib/backend/api_requests ` Find "Whisper" and instead of "key", provide your own api-key for openai whisper for transcriptions to work
 
-   ![CleanShot 2024-03-25 at 21 58 42](https://github.com/BasedHardware/Friend/assets/43514161/d0fb89d2-07fd-44e3-8563-68f938bb2319)
+   <img src="https://github.com/BasedHardware/Friend/assets/43514161/d0fb89d2-07fd-44e3-8563-68f938bb2319" alt="CleanShot 2024-03-25 at 21 58 42" width="400">
 
    - For AppStandalone, update variables in in .env.template file
 
@@ -95,44 +101,44 @@ Follow these steps to get started with your Friend. Note, we tested everything o
 - Don't have the device? [Clone this Flutterflow Project ](https://app.flutterflow.io/project/friend-0x9u40)
 - Have the wearable device? [Copy this Flutterflow Project](https://app.flutterflow.io/project/friend-share-19bk3d)
 
-### Install Firmware
+# Install Firmware
 
-1. [Download Arduino](https://www.arduino.cc/en/software)
-2. Run `cd src/BluetoothDeviceDriver ` in your home repository and Open Arduino .ino file, go to "Settings" and paste these 2 links in additional Boards Manager URLs
+Follow these steps to install the firmware:
 
-   ```
-   https://adafruit.github.io/arduino-board-index/package_adafruit_index.json
-   https://files.seeedstudio.com/arduino/package_seeeduino_boards_index.json
-   ```
+1. Set up nRF Connect by following the tutorial in this video: [https://youtu.be/EAJdOqsL9m8](https://youtu.be/EAJdOqsL9m8?feature=shared)
+
+2. In the nRF Connect Extension inside your VS Code, click "Open an existing application" and open the `firmware` folder from the root of this repo.
+
+   <img src="screenshots/vscode_extension.png" alt="VS Code Extension" width="200">
+
+3. In the application panel of the extension, click the "Add Build Configuration" icon.
 
-   ![IMAGE 2024-03-24 19:44:35](https://github.com/BasedHardware/friend/assets/43514161/f08cf422-8d30-4ffa-b61c-0e8ee4a0e685)
+   <img src="screenshots/addbuild.png" alt="Add Build Configuration" width="200">
 
-3. Go to Boards Manager and download these 2 Boards
+4. Choose the board as "xiao_ble_sense" and select the configuration as "prj.conf". Then, click "Build Configuration".
 
-   ![IMAGE 2024-03-24 19:46:49](https://github.com/BasedHardware/friend/assets/43514161/9c85a0c4-ee73-42ba-a75b-3f8fafa81cbe)
+   <img src="screenshots/build_settings.png" alt="Build Settings" width="400">
 
-4. Connect NRF52840 board via USB cable to your computer
-5. Go to Tools > Board >
-   ![IMAGE 2024-03-24 19:50:42](https://github.com/BasedHardware/friend/assets/43514161/065e794f-6e20-4f91-a6bf-1b43a5a3614e)
-   and select "Seeed nRF52 mbed-enabled Boards (you need board that has Sense)
+5. Once the build succeeds, you will find the `zephyr.uf2` file in the `firmware/build/zephyr` directory.
 
-   Also select Port (should be something that contains USB...)
-   ![IMAGE 2024-03-24 19:55:07](https://github.com/BasedHardware/friend/assets/43514161/0719de62-b58f-4ceb-85e2-d288916375c9)
+6. Double-click on the reset button of the device. The device will appear on your computer as a disk. Drag and drop the `zephyr.uf2` file into it.
+
+   > **Note:** On a Mac, you might see an error message after dropping the file, indicating that the process did not complete. This is just a Mac-specific error; the firmware is successfully uploaded.
 
-6. Go to Sketch => Include Library => Add .zip library and upload a library which you should download [from here](https://github.com/Seeed-Studio/Seeed_Arduino_Mic), make sure you download the `Seeed_Arduino_Mic` repository itself as a .zip file and not use the .zip from its releases section.
-7. Install Arduino BLE and Seeed Arduino LSM6DS3 libraries which can be found in Arduino's menu
-8. Click "Upload" and then open Serial Monitor to see logs
+   <img src="screenshots/pinout.jpg" alt="Pinout" width="300">
+
+That's it! You have successfully installed the firmware on your device.
 
 ### Testing Audio Recording on Your Computer
 
 Follow these steps to test audio recording on your computer using a Python script:
 
-1. Open your terminal and navigate to your home directory.
+1. Open your terminal and navigate to the project's root directory.
 
-2. Change to the "src" folder:
+2. Change to the "test" folder:
 
    ```
-   cd src
+   cd test
    ```
 
 3. Install the required Python modules:
@@ -147,63 +153,33 @@ Follow these steps to test audio recording on your computer using a Python scrip
    python local_laptop_client.py
    ```
 
-   This script will list the available audio devices and their corresponding IDs.
-
-5. Copy the ID of the desired audio device.
-
-6. Open the `local_laptop_client.py` file in a text editor and locate the following line:
-
-   ```python
-   DEVICE_ID = "564A72F4-4552-8CE8-719D-8D5CB2E5D43D"
-   ```
-
-   Replace `"564A72F4-4552-8CE8-719D-8D5CB2E5D43D"` with the ID you copied in step 5.
-
-7. Save the changes to the `local_laptop_client.py` file.
-
-8. Run the script again:
-
-   ```
-   python local_laptop_client.py
-   ```
-
-   or
-
-   ```
-   python3 local_laptop_client.py
-   ```
-
-9. You can now control the audio recording:
-   - Double-tap the device to start recording.
-   - Double-tap the device again to stop recording.
-
-The recorded audio files will be stored periodically in the `src/recordings` directory.
+The recorded audio files will be stored periodically in the `test/recordings` directory.
 
 That's it! You have now set up and tested audio recording on your computer.
 
 ### Assemble the device
 
 **Step 0:** Make sure you have bought everything from the buying guide above
 
-<img src="https://github.com/BasedHardware/Friend/assets/43514161/fdc7f8bd-6205-49a8-aa31-ea4ef6655ba4" width="300">
+<img src="https://github.com/BasedHardware/Friend/assets/43514161/fdc7f8bd-6205-49a8-aa31-ea4ef6655ba4" alt="Components" width="300">
 
 **Step 1:** You need to design the case using 3D printer. Find .stl file [here](https://github.com/BasedHardware/Friend/blob/main/3d-printing%20designs/Cover%20%2B%20Case.stl). If you don't know how to do it, send this file to someone who has a 3d printer
 
 **Step 2:**
 Solder everything together like on the picture below. using a soldering kit. Don't have it? buy [this one for $9](https://a.co/d/0XdthUV)
 
-![CleanShot 2024-03-28 at 17 01 53](https://github.com/BasedHardware/Friend/assets/43514161/c254668c-1662-412f-8b2c-05a97fb68419)
+<img src="https://github.com/BasedHardware/Friend/assets/43514161/c254668c-1662-412f-8b2c-05a97fb68419" alt="Soldering" width="400">
 
-<img src="https://github.com/BasedHardware/Friend/assets/43514161/5fe4cb81-eb64-41c6-b24c-e2da104b465e" width="300">
+<img src="https://github.com/BasedHardware/Friend/assets/43514161/5fe4cb81-eb64-41c6-b24c-e2da104b465e" alt="Soldered" width="300">
 
 **Step 3:**
 Fit everything in the case. Biggest hole is for the usb port. In my example, I put the battery first, then the board and then the switch, however it's not an ideal design. If you will figure out a better solution, please contribute!
 
-<img src="https://github.com/BasedHardware/Friend/assets/43514161/4abae04c-2477-4b9a-a74c-077a463f4c29" width="300">
+<img src="https://github.com/BasedHardware/Friend/assets/43514161/4abae04c-2477-4b9a-a74c-077a463f4c29" alt="Assembled" width="300">
 
 **Step 4:** Use hot glue to attach the lid to the case. You can also use a scotch tape first for testing purposes. Last, on the USB-port side, you'll find 2 small round holes. This is where the thread should go through.
 
-<img src="https://github.com/BasedHardware/Friend/assets/43514161/2ffcfbf4-6637-4bb6-89e5-bd75cf78eebd" width="200">
+<img src="https://github.com/BasedHardware/Friend/assets/43514161/2ffcfbf4-6637-4bb6-89e5-bd75cf78eebd" alt="Lid" width="200">
 
 Congratulations! you now have a fully working and assembled device!
 
@@ -236,4 +212,4 @@ Friend is available under dual licensing options:
 ### Choosing Your License
 
 - If you wish to contribute to or use Friend in open-source projects, you are free to do so under the terms of the GPL, as detailed in the LICENSE file.
-- If you require a commercial license for your project or enterprise, please contact us at [team@whomane.com](mailto:team@whomane.com) to discuss your needs and obtain licensing information.
+- If you require a commercial license for your project or enterprise, please contact us at [team@whomane.com](mailto:team@whomane.com) to discuss your needs and obtain licensing information.

firmware/CMakeLists.txt

@@ -0,0 +1,161 @@
+# SPDX-License-Identifier: Apache-2.0
+
+cmake_minimum_required(VERSION 3.20.0)
+find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
+project(nrf52840_nordic)
+enable_language(C ASM)
+
+target_sources(app PRIVATE
+    src/main.c
+    src/transport.c
+    src/mic.c
+    src/led.c
+    src/audio.c
+    src/codec.c
+    src/opus-1.2.1/A2NLSF.c
+    src/opus-1.2.1/CNG.c
+    src/opus-1.2.1/HP_variable_cutoff.c
+    src/opus-1.2.1/LPC_analysis_filter.c
+    src/opus-1.2.1/LPC_fit.c
+    src/opus-1.2.1/LPC_inv_pred_gain.c
+    src/opus-1.2.1/LP_variable_cutoff.c
+    src/opus-1.2.1/LTP_analysis_filter_FIX.c
+    src/opus-1.2.1/LTP_scale_ctrl_FIX.c
+    src/opus-1.2.1/NLSF2A.c
+    src/opus-1.2.1/NLSF_VQ.c
+    src/opus-1.2.1/NLSF_VQ_weights_laroia.c
+    src/opus-1.2.1/NLSF_decode.c
+    src/opus-1.2.1/NLSF_del_dec_quant.c
+    src/opus-1.2.1/NLSF_encode.c
+    src/opus-1.2.1/NLSF_stabilize.c
+    src/opus-1.2.1/NLSF_unpack.c
+    src/opus-1.2.1/NSQ.c
+    src/opus-1.2.1/NSQ_del_dec.c
+    src/opus-1.2.1/PLC.c
+    src/opus-1.2.1/VAD.c
+    src/opus-1.2.1/VQ_WMat_EC.c
+    src/opus-1.2.1/ana_filt_bank_1.c
+    src/opus-1.2.1/analysis.c
+    src/opus-1.2.1/apply_sine_window_FIX.c
+    src/opus-1.2.1/autocorr_FIX.c
+    src/opus-1.2.1/bands.c
+    src/opus-1.2.1/biquad_alt.c
+    src/opus-1.2.1/burg_modified_FIX.c
+    src/opus-1.2.1/bwexpander.c
+    src/opus-1.2.1/bwexpander_32.c
+    src/opus-1.2.1/celt.c
+    src/opus-1.2.1/celt_decoder.c
+    src/opus-1.2.1/celt_encoder.c
+    src/opus-1.2.1/celt_lpc.c
+    src/opus-1.2.1/arm/celt_pitch_xcorr_arm_gcc.s
+    src/opus-1.2.1/check_control_input.c
+    src/opus-1.2.1/code_signs.c
+    src/opus-1.2.1/control_SNR.c
+    src/opus-1.2.1/control_audio_bandwidth.c
+    src/opus-1.2.1/control_codec.c
+    src/opus-1.2.1/corrMatrix_FIX.c
+    src/opus-1.2.1/cwrs.c
+    src/opus-1.2.1/debug.c
+    src/opus-1.2.1/dec_API.c
+    src/opus-1.2.1/decode_core.c
+    src/opus-1.2.1/decode_frame.c
+    src/opus-1.2.1/decode_indices.c
+    src/opus-1.2.1/decode_parameters.c
+    src/opus-1.2.1/decode_pitch.c
+    src/opus-1.2.1/decode_pulses.c
+    src/opus-1.2.1/decoder_set_fs.c
+    src/opus-1.2.1/enc_API.c
+    src/opus-1.2.1/encode_frame_FIX.c
+    src/opus-1.2.1/encode_indices.c
+    src/opus-1.2.1/encode_pulses.c
+    src/opus-1.2.1/entcode.c
+    src/opus-1.2.1/entdec.c
+    src/opus-1.2.1/entenc.c
+    src/opus-1.2.1/find_LPC_FIX.c
+    src/opus-1.2.1/find_LTP_FIX.c
+    src/opus-1.2.1/find_pitch_lags_FIX.c
+    src/opus-1.2.1/find_pred_coefs_FIX.c
+    src/opus-1.2.1/gain_quant.c
+    src/opus-1.2.1/init_decoder.c
+    src/opus-1.2.1/init_encoder.c
+    src/opus-1.2.1/inner_prod_aligned.c
+    src/opus-1.2.1/interpolate.c
+    src/opus-1.2.1/k2a_FIX.c
+    src/opus-1.2.1/k2a_Q16_FIX.c
+    src/opus-1.2.1/kiss_fft.c
+    src/opus-1.2.1/laplace.c
+    src/opus-1.2.1/lin2log.c
+    src/opus-1.2.1/log2lin.c
+    src/opus-1.2.1/mathops.c
+    src/opus-1.2.1/mdct.c
+    src/opus-1.2.1/mlp.c
+    src/opus-1.2.1/mlp_data.c
+    src/opus-1.2.1/modes.c
+    src/opus-1.2.1/noise_shape_analysis_FIX.c
+    src/opus-1.2.1/opus.c
+    src/opus-1.2.1/opus_decoder.c
+    src/opus-1.2.1/opus_encoder.c
+    src/opus-1.2.1/opus_multistream.c
+    src/opus-1.2.1/opus_multistream_decoder.c
+    src/opus-1.2.1/opus_multistream_encoder.c
+    src/opus-1.2.1/pitch.c
+    src/opus-1.2.1/pitch_analysis_core_FIX.c
+    src/opus-1.2.1/pitch_est_tables.c
+    src/opus-1.2.1/process_NLSFs.c
+    src/opus-1.2.1/process_gains_FIX.c
+    src/opus-1.2.1/quant_LTP_gains.c
+    src/opus-1.2.1/quant_bands.c
+    src/opus-1.2.1/rate.c
+    src/opus-1.2.1/regularize_correlations_FIX.c
+    src/opus-1.2.1/repacketizer.c
+    src/opus-1.2.1/resampler.c
+    src/opus-1.2.1/resampler_down2.c
+    src/opus-1.2.1/resampler_down2_3.c
+    src/opus-1.2.1/resampler_private_AR2.c
+    src/opus-1.2.1/resampler_private_IIR_FIR.c
+    src/opus-1.2.1/resampler_private_down_FIR.c
+    src/opus-1.2.1/resampler_private_up2_HQ.c
+    src/opus-1.2.1/resampler_rom.c
+    src/opus-1.2.1/residual_energy16_FIX.c
+    src/opus-1.2.1/residual_energy_FIX.c
+    src/opus-1.2.1/schur64_FIX.c
+    src/opus-1.2.1/schur_FIX.c
+    src/opus-1.2.1/shell_coder.c
+    src/opus-1.2.1/sigm_Q15.c
+    src/opus-1.2.1/sort.c
+    src/opus-1.2.1/stereo_LR_to_MS.c
+    src/opus-1.2.1/stereo_MS_to_LR.c
+    src/opus-1.2.1/stereo_decode_pred.c
+    src/opus-1.2.1/stereo_encode_pred.c
+    src/opus-1.2.1/stereo_find_predictor.c
+    src/opus-1.2.1/stereo_quant_pred.c
+    src/opus-1.2.1/sum_sqr_shift.c
+    src/opus-1.2.1/table_LSF_cos.c
+    src/opus-1.2.1/tables_LTP.c
+    src/opus-1.2.1/tables_NLSF_CB_NB_MB.c
+    src/opus-1.2.1/tables_NLSF_CB_WB.c
+    src/opus-1.2.1/tables_gain.c
+    src/opus-1.2.1/tables_other.c
+    src/opus-1.2.1/tables_pitch_lag.c
+    src/opus-1.2.1/tables_pulses_per_block.c
+    src/opus-1.2.1/vector_ops_FIX.c
+    src/opus-1.2.1/vq.c
+    src/opus-1.2.1/warped_autocorrelation_FIX.c
+    src/opus-1.2.1/arm/celt_pitch_xcorr_arm_gcc.s
+)
+
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DARM_MATH_CM4")
+# set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DVAR_ARRAYS")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOPUS_ARM_ASM")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOPUS_ARM_INLINE_ASM")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOPUS_ARM_INLINE_EDSP")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOPUS_ARM_INLINE_MEDIA")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOPUS_ARM_MAY_HAVE_EDSP")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOPUS_ARM_PRESUME_EDSP")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOPUS_BUILD")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DUSE_ALLOCA")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DFIXED_POINT -DDISABLE_FLOAT_API")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DHAVE_CONFIG_H")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DHAVE_ALLOCA_H")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fsingle-precision-constant") # A lot of constants are written as doubles
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DHAVE_LRINT -DHAVE_LRINTF")

firmware/client.py

@@ -9,11 +9,11 @@
 import struct
 from scipy.signal import stft, istft
 
-DEVICE_NAME = "Super"
+DEVICE_NAME = "Friend"
 SERVICE_UUID = "19B10000-E8F2-537E-4F6C-D104768A1214"
 CHARACTERISTIC_UUID = "19B10001-E8F2-537E-4F6C-D104768A1214"
 
-CODEC = "mulaw"  # "pcm" or "mulaw" 
+CODEC = "pcm"  # "pcm" or "mulaw" 
 SAMPLE_RATE = 8000  # Sample rate for the audio
 SAMPLE_WIDTH = 2  # 16-bit audio
 CHANNELS = 1  # Mono audio

firmware/prj.conf

@@ -11,7 +11,7 @@ CONFIG_NRFX_PDM=y
 
 CONFIG_BT=y
 CONFIG_BT_PERIPHERAL=y
-CONFIG_BT_DEVICE_NAME="Super"
+CONFIG_BT_DEVICE_NAME="Friend"
 CONFIG_BT_MAX_CONN=1
 CONFIG_BT_MAX_PAIRED=1
 CONFIG_BT_DEVICE_APPEARANCE=22

firmware/src/transport.c

@@ -37,7 +37,7 @@ static struct bt_gatt_service audio_service = BT_GATT_SERVICE(attrs);
 // Advertisement data
 static const struct bt_data bt_ad[] = {
     BT_DATA_BYTES(BT_DATA_FLAGS, (BT_LE_AD_GENERAL | BT_LE_AD_NO_BREDR)),
-    BT_DATA(BT_DATA_NAME_COMPLETE, "Super", sizeof("Super") - 1),
+    BT_DATA(BT_DATA_NAME_COMPLETE, "Friend", sizeof("Friend") - 1),
 };
 
 // Scan response data