diff --git a/README.md b/README.md
index 93ef3b5cc72..0f37f39ead9 100644
--- a/README.md
+++ b/README.md
@@ -1,9 +1,15 @@
# Friend: Open-Source AI Wearable with 24h+ on single charge
-| Assembled | Disassembled |
-| :-------------------------------------------------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------------------------------------: |
-| 
| 
|
-
+
+
+ |
+ |
+
+
+ | Assembled |
+ Disassembled |
+
+
[](https://discord.gg/kEXXsnb5b3)
[](https://opensource.org/license/agpl-v3)
@@ -67,7 +73,7 @@ There are 2 different apps in these repositories located in different branches a
## Getting Started
-Follow these steps to get started with your Friend. Note, we tested everything on a mac + iphone. Currently trying to make it work for Android
+Follow these steps to get started with your Friend.
### Install the app
@@ -80,7 +86,7 @@ Follow these steps to get started with your Friend. Note, we tested everything o
- For AppWithWearable, open file api_calls.dart located in `AppWithWearable/lib/backend/api_requests ` Find "Whisper" and instead of "key", provide your own api-key for openai whisper for transcriptions to work
- 
+ 
- For AppStandalone, update variables in in .env.template file
@@ -95,44 +101,44 @@ Follow these steps to get started with your Friend. Note, we tested everything o
- Don't have the device? [Clone this Flutterflow Project ](https://app.flutterflow.io/project/friend-0x9u40)
- Have the wearable device? [Copy this Flutterflow Project](https://app.flutterflow.io/project/friend-share-19bk3d)
-### Install Firmware
+# Install Firmware
-1. [Download Arduino](https://www.arduino.cc/en/software)
-2. Run `cd src/BluetoothDeviceDriver ` in your home repository and Open Arduino .ino file, go to "Settings" and paste these 2 links in additional Boards Manager URLs
+Follow these steps to install the firmware:
- ```
- https://adafruit.github.io/arduino-board-index/package_adafruit_index.json
- https://files.seeedstudio.com/arduino/package_seeeduino_boards_index.json
- ```
+1. Set up nRF Connect by following the tutorial in this video: [https://youtu.be/EAJdOqsL9m8](https://youtu.be/EAJdOqsL9m8?feature=shared)
+
+2. In the nRF Connect Extension inside your VS Code, click "Open an existing application" and open the `firmware` folder from the root of this repo.
+
+ 
+
+3. In the application panel of the extension, click the "Add Build Configuration" icon.
- 
+ 
-3. Go to Boards Manager and download these 2 Boards
+4. Choose the board as "xiao_ble_sense" and select the configuration as "prj.conf". Then, click "Build Configuration".
- 
+ 
-4. Connect NRF52840 board via USB cable to your computer
-5. Go to Tools > Board >
- 
- and select "Seeed nRF52 mbed-enabled Boards (you need board that has Sense)
+5. Once the build succeeds, you will find the `zephyr.uf2` file in the `firmware/build/zephyr` directory.
- Also select Port (should be something that contains USB...)
- 
+6. Double-click on the reset button of the device. The device will appear on your computer as a disk. Drag and drop the `zephyr.uf2` file into it.
+
+ > **Note:** On a Mac, you might see an error message after dropping the file, indicating that the process did not complete. This is just a Mac-specific error; the firmware is successfully uploaded.
-6. Go to Sketch => Include Library => Add .zip library and upload a library which you should download [from here](https://github.com/Seeed-Studio/Seeed_Arduino_Mic), make sure you download the `Seeed_Arduino_Mic` repository itself as a .zip file and not use the .zip from its releases section.
-7. Install Arduino BLE and Seeed Arduino LSM6DS3 libraries which can be found in Arduino's menu
-8. Click "Upload" and then open Serial Monitor to see logs
+ 
+
+That's it! You have successfully installed the firmware on your device.
### Testing Audio Recording on Your Computer
Follow these steps to test audio recording on your computer using a Python script:
-1. Open your terminal and navigate to your home directory.
+1. Open your terminal and navigate to the project's root directory.
-2. Change to the "src" folder:
+2. Change to the "test" folder:
```
- cd src
+ cd test
```
3. Install the required Python modules:
@@ -147,37 +153,7 @@ Follow these steps to test audio recording on your computer using a Python scrip
python local_laptop_client.py
```
- This script will list the available audio devices and their corresponding IDs.
-
-5. Copy the ID of the desired audio device.
-
-6. Open the `local_laptop_client.py` file in a text editor and locate the following line:
-
- ```python
- DEVICE_ID = "564A72F4-4552-8CE8-719D-8D5CB2E5D43D"
- ```
-
- Replace `"564A72F4-4552-8CE8-719D-8D5CB2E5D43D"` with the ID you copied in step 5.
-
-7. Save the changes to the `local_laptop_client.py` file.
-
-8. Run the script again:
-
- ```
- python local_laptop_client.py
- ```
-
- or
-
- ```
- python3 local_laptop_client.py
- ```
-
-9. You can now control the audio recording:
- - Double-tap the device to start recording.
- - Double-tap the device again to stop recording.
-
-The recorded audio files will be stored periodically in the `src/recordings` directory.
+The recorded audio files will be stored periodically in the `test/recordings` directory.
That's it! You have now set up and tested audio recording on your computer.
@@ -185,25 +161,25 @@ That's it! You have now set up and tested audio recording on your computer.
**Step 0:** Make sure you have bought everything from the buying guide above
-
+
**Step 1:** You need to design the case using 3D printer. Find .stl file [here](https://github.com/BasedHardware/Friend/blob/main/3d-printing%20designs/Cover%20%2B%20Case.stl). If you don't know how to do it, send this file to someone who has a 3d printer
**Step 2:**
Solder everything together like on the picture below. using a soldering kit. Don't have it? buy [this one for $9](https://a.co/d/0XdthUV)
-
+
-
+
**Step 3:**
Fit everything in the case. Biggest hole is for the usb port. In my example, I put the battery first, then the board and then the switch, however it's not an ideal design. If you will figure out a better solution, please contribute!
-
+
**Step 4:** Use hot glue to attach the lid to the case. You can also use a scotch tape first for testing purposes. Last, on the USB-port side, you'll find 2 small round holes. This is where the thread should go through.
-
+
Congratulations! you now have a fully working and assembled device!
@@ -236,4 +212,4 @@ Friend is available under dual licensing options:
### Choosing Your License
- If you wish to contribute to or use Friend in open-source projects, you are free to do so under the terms of the GPL, as detailed in the LICENSE file.
-- If you require a commercial license for your project or enterprise, please contact us at [team@whomane.com](mailto:team@whomane.com) to discuss your needs and obtain licensing information.
+- If you require a commercial license for your project or enterprise, please contact us at [team@whomane.com](mailto:team@whomane.com) to discuss your needs and obtain licensing information.
\ No newline at end of file
diff --git a/Screenshots/addbuild.png b/Screenshots/addbuild.png
new file mode 100644
index 00000000000..a12f6215161
Binary files /dev/null and b/Screenshots/addbuild.png differ
diff --git a/Screenshots/build_settings.png b/Screenshots/build_settings.png
new file mode 100644
index 00000000000..b8cde502188
Binary files /dev/null and b/Screenshots/build_settings.png differ
diff --git a/Screenshots/pinout.jpg b/Screenshots/pinout.jpg
new file mode 100644
index 00000000000..295969b901f
Binary files /dev/null and b/Screenshots/pinout.jpg differ
diff --git a/Screenshots/vscode_extension.png b/Screenshots/vscode_extension.png
new file mode 100644
index 00000000000..1cdcd331c76
Binary files /dev/null and b/Screenshots/vscode_extension.png differ
diff --git a/firmware/CMakeLists.txt b/firmware/CMakeLists.txt
new file mode 100644
index 00000000000..dfe7aa159c8
--- /dev/null
+++ b/firmware/CMakeLists.txt
@@ -0,0 +1,161 @@
+# SPDX-License-Identifier: Apache-2.0
+
+cmake_minimum_required(VERSION 3.20.0)
+find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
+project(nrf52840_nordic)
+enable_language(C ASM)
+
+target_sources(app PRIVATE
+ src/main.c
+ src/transport.c
+ src/mic.c
+ src/led.c
+ src/audio.c
+ src/codec.c
+ src/opus-1.2.1/A2NLSF.c
+ src/opus-1.2.1/CNG.c
+ src/opus-1.2.1/HP_variable_cutoff.c
+ src/opus-1.2.1/LPC_analysis_filter.c
+ src/opus-1.2.1/LPC_fit.c
+ src/opus-1.2.1/LPC_inv_pred_gain.c
+ src/opus-1.2.1/LP_variable_cutoff.c
+ src/opus-1.2.1/LTP_analysis_filter_FIX.c
+ src/opus-1.2.1/LTP_scale_ctrl_FIX.c
+ src/opus-1.2.1/NLSF2A.c
+ src/opus-1.2.1/NLSF_VQ.c
+ src/opus-1.2.1/NLSF_VQ_weights_laroia.c
+ src/opus-1.2.1/NLSF_decode.c
+ src/opus-1.2.1/NLSF_del_dec_quant.c
+ src/opus-1.2.1/NLSF_encode.c
+ src/opus-1.2.1/NLSF_stabilize.c
+ src/opus-1.2.1/NLSF_unpack.c
+ src/opus-1.2.1/NSQ.c
+ src/opus-1.2.1/NSQ_del_dec.c
+ src/opus-1.2.1/PLC.c
+ src/opus-1.2.1/VAD.c
+ src/opus-1.2.1/VQ_WMat_EC.c
+ src/opus-1.2.1/ana_filt_bank_1.c
+ src/opus-1.2.1/analysis.c
+ src/opus-1.2.1/apply_sine_window_FIX.c
+ src/opus-1.2.1/autocorr_FIX.c
+ src/opus-1.2.1/bands.c
+ src/opus-1.2.1/biquad_alt.c
+ src/opus-1.2.1/burg_modified_FIX.c
+ src/opus-1.2.1/bwexpander.c
+ src/opus-1.2.1/bwexpander_32.c
+ src/opus-1.2.1/celt.c
+ src/opus-1.2.1/celt_decoder.c
+ src/opus-1.2.1/celt_encoder.c
+ src/opus-1.2.1/celt_lpc.c
+ src/opus-1.2.1/arm/celt_pitch_xcorr_arm_gcc.s
+ src/opus-1.2.1/check_control_input.c
+ src/opus-1.2.1/code_signs.c
+ src/opus-1.2.1/control_SNR.c
+ src/opus-1.2.1/control_audio_bandwidth.c
+ src/opus-1.2.1/control_codec.c
+ src/opus-1.2.1/corrMatrix_FIX.c
+ src/opus-1.2.1/cwrs.c
+ src/opus-1.2.1/debug.c
+ src/opus-1.2.1/dec_API.c
+ src/opus-1.2.1/decode_core.c
+ src/opus-1.2.1/decode_frame.c
+ src/opus-1.2.1/decode_indices.c
+ src/opus-1.2.1/decode_parameters.c
+ src/opus-1.2.1/decode_pitch.c
+ src/opus-1.2.1/decode_pulses.c
+ src/opus-1.2.1/decoder_set_fs.c
+ src/opus-1.2.1/enc_API.c
+ src/opus-1.2.1/encode_frame_FIX.c
+ src/opus-1.2.1/encode_indices.c
+ src/opus-1.2.1/encode_pulses.c
+ src/opus-1.2.1/entcode.c
+ src/opus-1.2.1/entdec.c
+ src/opus-1.2.1/entenc.c
+ src/opus-1.2.1/find_LPC_FIX.c
+ src/opus-1.2.1/find_LTP_FIX.c
+ src/opus-1.2.1/find_pitch_lags_FIX.c
+ src/opus-1.2.1/find_pred_coefs_FIX.c
+ src/opus-1.2.1/gain_quant.c
+ src/opus-1.2.1/init_decoder.c
+ src/opus-1.2.1/init_encoder.c
+ src/opus-1.2.1/inner_prod_aligned.c
+ src/opus-1.2.1/interpolate.c
+ src/opus-1.2.1/k2a_FIX.c
+ src/opus-1.2.1/k2a_Q16_FIX.c
+ src/opus-1.2.1/kiss_fft.c
+ src/opus-1.2.1/laplace.c
+ src/opus-1.2.1/lin2log.c
+ src/opus-1.2.1/log2lin.c
+ src/opus-1.2.1/mathops.c
+ src/opus-1.2.1/mdct.c
+ src/opus-1.2.1/mlp.c
+ src/opus-1.2.1/mlp_data.c
+ src/opus-1.2.1/modes.c
+ src/opus-1.2.1/noise_shape_analysis_FIX.c
+ src/opus-1.2.1/opus.c
+ src/opus-1.2.1/opus_decoder.c
+ src/opus-1.2.1/opus_encoder.c
+ src/opus-1.2.1/opus_multistream.c
+ src/opus-1.2.1/opus_multistream_decoder.c
+ src/opus-1.2.1/opus_multistream_encoder.c
+ src/opus-1.2.1/pitch.c
+ src/opus-1.2.1/pitch_analysis_core_FIX.c
+ src/opus-1.2.1/pitch_est_tables.c
+ src/opus-1.2.1/process_NLSFs.c
+ src/opus-1.2.1/process_gains_FIX.c
+ src/opus-1.2.1/quant_LTP_gains.c
+ src/opus-1.2.1/quant_bands.c
+ src/opus-1.2.1/rate.c
+ src/opus-1.2.1/regularize_correlations_FIX.c
+ src/opus-1.2.1/repacketizer.c
+ src/opus-1.2.1/resampler.c
+ src/opus-1.2.1/resampler_down2.c
+ src/opus-1.2.1/resampler_down2_3.c
+ src/opus-1.2.1/resampler_private_AR2.c
+ src/opus-1.2.1/resampler_private_IIR_FIR.c
+ src/opus-1.2.1/resampler_private_down_FIR.c
+ src/opus-1.2.1/resampler_private_up2_HQ.c
+ src/opus-1.2.1/resampler_rom.c
+ src/opus-1.2.1/residual_energy16_FIX.c
+ src/opus-1.2.1/residual_energy_FIX.c
+ src/opus-1.2.1/schur64_FIX.c
+ src/opus-1.2.1/schur_FIX.c
+ src/opus-1.2.1/shell_coder.c
+ src/opus-1.2.1/sigm_Q15.c
+ src/opus-1.2.1/sort.c
+ src/opus-1.2.1/stereo_LR_to_MS.c
+ src/opus-1.2.1/stereo_MS_to_LR.c
+ src/opus-1.2.1/stereo_decode_pred.c
+ src/opus-1.2.1/stereo_encode_pred.c
+ src/opus-1.2.1/stereo_find_predictor.c
+ src/opus-1.2.1/stereo_quant_pred.c
+ src/opus-1.2.1/sum_sqr_shift.c
+ src/opus-1.2.1/table_LSF_cos.c
+ src/opus-1.2.1/tables_LTP.c
+ src/opus-1.2.1/tables_NLSF_CB_NB_MB.c
+ src/opus-1.2.1/tables_NLSF_CB_WB.c
+ src/opus-1.2.1/tables_gain.c
+ src/opus-1.2.1/tables_other.c
+ src/opus-1.2.1/tables_pitch_lag.c
+ src/opus-1.2.1/tables_pulses_per_block.c
+ src/opus-1.2.1/vector_ops_FIX.c
+ src/opus-1.2.1/vq.c
+ src/opus-1.2.1/warped_autocorrelation_FIX.c
+ src/opus-1.2.1/arm/celt_pitch_xcorr_arm_gcc.s
+)
+
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DARM_MATH_CM4")
+# set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DVAR_ARRAYS")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOPUS_ARM_ASM")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOPUS_ARM_INLINE_ASM")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOPUS_ARM_INLINE_EDSP")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOPUS_ARM_INLINE_MEDIA")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOPUS_ARM_MAY_HAVE_EDSP")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOPUS_ARM_PRESUME_EDSP")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOPUS_BUILD")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DUSE_ALLOCA")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DFIXED_POINT -DDISABLE_FLOAT_API")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DHAVE_CONFIG_H")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DHAVE_ALLOCA_H")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fsingle-precision-constant") # A lot of constants are written as doubles
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DHAVE_LRINT -DHAVE_LRINTF")
\ No newline at end of file
diff --git a/firmware/client.py b/firmware/client.py
index d3ed244f1b2..97b8f2dbfff 100644
--- a/firmware/client.py
+++ b/firmware/client.py
@@ -9,11 +9,11 @@
import struct
from scipy.signal import stft, istft
-DEVICE_NAME = "Super"
+DEVICE_NAME = "Friend"
SERVICE_UUID = "19B10000-E8F2-537E-4F6C-D104768A1214"
CHARACTERISTIC_UUID = "19B10001-E8F2-537E-4F6C-D104768A1214"
-CODEC = "mulaw" # "pcm" or "mulaw"
+CODEC = "pcm" # "pcm" or "mulaw"
SAMPLE_RATE = 8000 # Sample rate for the audio
SAMPLE_WIDTH = 2 # 16-bit audio
CHANNELS = 1 # Mono audio
diff --git a/firmware/prj.conf b/firmware/prj.conf
index 1147ea4866b..5db4e9ea606 100644
--- a/firmware/prj.conf
+++ b/firmware/prj.conf
@@ -11,7 +11,7 @@ CONFIG_NRFX_PDM=y
CONFIG_BT=y
CONFIG_BT_PERIPHERAL=y
-CONFIG_BT_DEVICE_NAME="Super"
+CONFIG_BT_DEVICE_NAME="Friend"
CONFIG_BT_MAX_CONN=1
CONFIG_BT_MAX_PAIRED=1
CONFIG_BT_DEVICE_APPEARANCE=22
diff --git a/firmware/src/transport.c b/firmware/src/transport.c
index 353e34a8465..bffbaeb0b39 100644
--- a/firmware/src/transport.c
+++ b/firmware/src/transport.c
@@ -37,7 +37,7 @@ static struct bt_gatt_service audio_service = BT_GATT_SERVICE(attrs);
// Advertisement data
static const struct bt_data bt_ad[] = {
BT_DATA_BYTES(BT_DATA_FLAGS, (BT_LE_AD_GENERAL | BT_LE_AD_NO_BREDR)),
- BT_DATA(BT_DATA_NAME_COMPLETE, "Super", sizeof("Super") - 1),
+ BT_DATA(BT_DATA_NAME_COMPLETE, "Friend", sizeof("Friend") - 1),
};
// Scan response data
diff --git a/src/BLE_Audio_Stream_NRF52840 b/src/BLE_Audio_Stream_NRF52840
deleted file mode 160000
index b668822272a..00000000000
--- a/src/BLE_Audio_Stream_NRF52840
+++ /dev/null
@@ -1 +0,0 @@
-Subproject commit b668822272ade958bf57b1aad576aea262782c49
diff --git a/src/BluetoothDeviceDriver/BluetoothDeviceDriver.ino b/src/BluetoothDeviceDriver/BluetoothDeviceDriver.ino
deleted file mode 100644
index 67c9f0de35f..00000000000
--- a/src/BluetoothDeviceDriver/BluetoothDeviceDriver.ino
+++ /dev/null
@@ -1,176 +0,0 @@
-#include
-#include // make sure to use the library "Seeed arduino LSM6DS3"
-#include
-
-// Settings
-#define DEBUG 1 // Enable pin pulse during ISR
-#define SAMPLES 4000 // Changed to 4000
-#define CHUNK_SIZE 200
-#define BUFFER_SIZE (SAMPLES * 4) // Doubled the buffer size to accommodate continuous recording
-#define int2Pin PIN_LSM6DS3TR_C_INT1
-//#define DOUBLE_TAP_ENABLED // This enables double tap and usage of IMU
-
-mic_config_t mic_config{
- .channel_cnt = 1,
- .sampling_rate = 16000, // Keep the sampling rate at 16000
- .buf_size = 16000, // Use the larger buffer size
- .debug_pin = LED_BUILTIN // Toggles each DAC ISR (if DEBUG is set to 1)
-};
-
-LSM6DS3 gyro(I2C_MODE, 0x6A); // gyro init
-NRF52840_ADC_Class Mic(&mic_config);
-
-uint16_t recording_buf[BUFFER_SIZE];
-volatile uint32_t recording_idx = 0;
-volatile uint32_t read_idx = 0;
-volatile bool recording = false;
-bool isConnected = false;
-
-uint8_t tapCount = 0; // Amount of received taps
-uint8_t prevTapCount = 0; // Tap Counter from last loop
-
-BLEService audioService("19B10000-E8F2-537E-4F6C-D104768A1214");
-BLECharacteristic audioCharacteristic("19B10001-E8F2-537E-4F6C-D104768A1214", BLERead | BLENotify, CHUNK_SIZE * sizeof(int16_t));
-
-void setup() {
- pinMode(int2Pin, INPUT);
- pinMode(LED_BUILTIN, OUTPUT);
- pinMode(LEDR, OUTPUT);
- pinMode(LED_BLUE, OUTPUT);
- pinMode(LED_GREEN, OUTPUT);
- Serial.begin(115200);
-
- unsigned long startMillis = millis();
- while (!Serial && millis() - startMillis < 500) {
- delay(10); // Short delay to prevent hanging in a tight loop
- }
-
- Mic.set_callback(audio_rec_callback);
- if (!Mic.begin()) {
- Serial.println("Mic initialization failed");
- setLedRGB(true, false, false);
- while (1)
- ;
- }
- Serial.println("Mic initialization done.");
- setLedRGB(false, true, false);
-
- if (!BLE.begin()) {
- Serial.println("Starting Bluetooth® Low Energy module failed!");
- setLedRGB(true, false, false);
- while (1)
- ;
- }
-#ifdef DOUBLE_TAP_ENABLED
- if (gyro.begin() != 0) {
- Serial.println("gyro error");
- setLedRGB(true, false, false);
- while (1)
- ;
- }
- Serial.println("Gyro initialization done.");
-
- setupDoubleTap();
-
- attachInterrupt(digitalPinToInterrupt(int2Pin), tapCallback, RISING);
-#endif
- setLedRGB(false, true, false);
- BLE.setLocalName("AudioRecorder");
-
- BLE.setAdvertisedService(audioService);
- audioService.addCharacteristic(audioCharacteristic);
- BLE.addService(audioService);
- BLE.advertise();
-
- // Print device address
- Serial.print("Device Address: ");
- Serial.println(BLE.address());
-
- Serial.println("BLE Audio Recorder");
-}
-
-void loop() {
- BLEDevice central = BLE.central();
-
- if (central && !isConnected) {
- Serial.print("Connected to central: ");
- Serial.println(central.address());
- isConnected = true;
- Serial.println("Type 'rec' to start recording");
- }
-
-#ifdef DOUBLE_TAP_ENABLED
- if (tapCount > prevTapCount) {
- Serial.println("Double tapped!");
- recording = !recording; // Toggle the recording state
-
- if (recording) {
- setLedRGB(true, false, true); // Purple color for recording
- Serial.println("Recording started");
- } else {
- setLedRGB(false, true, false); // Green color for stopped
- Serial.println("Recording stopped");
- }
- }
-
- prevTapCount = tapCount;
-#endif
-
- if (Serial.available()) {
- String resp = Serial.readStringUntil('\n');
- if (resp == "rec" && !recording) {
- recording = true;
- setLedRGB(false, false, true);
- Serial.println("Recording started");
- } else if (resp == "stop" && recording) {
- recording = false;
- setLedRGB(false, true, false);
- Serial.println("Recording stopped");
- }
- }
-
- if (recording) {
- uint32_t available_samples = (recording_idx + BUFFER_SIZE - read_idx) % BUFFER_SIZE;
- if (available_samples >= CHUNK_SIZE) {
-#ifdef DEBUG
- Serial.println("Sending " + String(available_samples) + " samples");
-#endif
- uint16_t chunk[CHUNK_SIZE];
- for (int i = 0; i < CHUNK_SIZE; i++) {
- chunk[i] = recording_buf[(read_idx + i) % BUFFER_SIZE];
- }
- audioCharacteristic.writeValue(chunk, sizeof(chunk));
- read_idx = (read_idx + CHUNK_SIZE) % BUFFER_SIZE;
- delay(20);
- }
- }
-}
-
-static void audio_rec_callback(uint16_t *buf, uint32_t buf_len) {
- if (recording) {
- for (uint32_t i = 0; i < buf_len; i += 4) {
- recording_buf[recording_idx] = buf[i];
- recording_idx = (recording_idx + 1) % BUFFER_SIZE;
- }
- }
-}
-
-void setupDoubleTap() {
- // Double Tap Config
- gyro.writeRegister(LSM6DS3_ACC_GYRO_CTRL1_XL, 0x60); //* Acc = 416Hz (High-Performance mode)// Turn on the accelerometer
- gyro.writeRegister(LSM6DS3_ACC_GYRO_TAP_CFG1, 0x8E); // INTERRUPTS_ENABLE, SLOPE_FDS// Enable interrupts and tap detection on X, Y, Z-axis
- gyro.writeRegister(LSM6DS3_ACC_GYRO_TAP_THS_6D, 0x85); // Set tap threshold 8C
- gyro.writeRegister(LSM6DS3_ACC_GYRO_INT_DUR2, 0x7F); // Set Duration, Quiet and Shock time windows 7F
- gyro.writeRegister(LSM6DS3_ACC_GYRO_WAKE_UP_THS, 0x80); // Single & double-tap enabled (SINGLE_DOUBLE_TAP = 1)
- gyro.writeRegister(LSM6DS3_ACC_GYRO_MD1_CFG, 0x08); // Double-tap interrupt driven to INT1 pin
-}
-
-void tapCallback() {
- tapCount++;
-}
-
-void setLedRGB(bool red, bool green, bool blue) {
- digitalWrite(LEDB, blue ? LOW : HIGH); // Blue ON when blue is true
- digitalWrite(LEDG, green ? LOW : HIGH); // Green ON when green is true
- digitalWrite(LEDR, red ? LOW : HIGH); // Red ON when red is true
-}
\ No newline at end of file
diff --git a/src/requirements.txt b/src/requirements.txt
deleted file mode 100644
index ce84222e812..00000000000
--- a/src/requirements.txt
+++ /dev/null
@@ -1,3 +0,0 @@
-bleak==0.21.1
-numpy==1.26.4
-scipy==1.12.0
\ No newline at end of file
diff --git a/src/local_laptop_client.py b/test/local_laptop_client.py
similarity index 53%
rename from src/local_laptop_client.py
rename to test/local_laptop_client.py
index bf91521365f..97b8f2dbfff 100644
--- a/src/local_laptop_client.py
+++ b/test/local_laptop_client.py
@@ -1,30 +1,49 @@
import asyncio
import bleak
-from bleak import BleakClient, BleakScanner
+from bleak import BleakClient
import wave
from datetime import datetime
import numpy as np
import time
import os
+import struct
from scipy.signal import stft, istft
-DEVICE_ID = "564A72F4-4552-8CE8-719D-8D5CB2E5D43D" # NOTE: You will have to update this ID with your devices bluetooth id
+DEVICE_NAME = "Friend"
SERVICE_UUID = "19B10000-E8F2-537E-4F6C-D104768A1214"
CHARACTERISTIC_UUID = "19B10001-E8F2-537E-4F6C-D104768A1214"
-SAMPLE_RATE = 4000 # Sample rate for the audio
+CODEC = "pcm" # "pcm" or "mulaw"
+SAMPLE_RATE = 8000 # Sample rate for the audio
SAMPLE_WIDTH = 2 # 16-bit audio
CHANNELS = 1 # Mono audio
-CAPTURE_TIME = 30 # Time to capture audio in seconds
+CAPTURE_TIME = 10 # Time to capture audio in seconds
+
+def ulaw2linear(ulaw_byte):
+ """Convert a µ-law byte to a 16-bit linear PCM value."""
+ EXPONENT_LUT = [0, 132, 396, 924, 1980, 4092, 8316, 16764]
+ ulaw_byte = ~ulaw_byte
+ sign = (ulaw_byte & 0x80)
+ exponent = (ulaw_byte >> 4) & 0x07
+ mantissa = ulaw_byte & 0x0F
+ sample = EXPONENT_LUT[exponent] + (mantissa << (exponent + 3))
+ if sign != 0:
+ sample = -sample
+
+ return sample
+
+def ulaw_bytes_to_pcm16(ulaw_data):
+ """Convert a sequence of µ-law encoded bytes to a list of 16-bit PCM values."""
+ return [ulaw2linear(byte) for byte in ulaw_data]
async def main():
print("Discovering AudioRecorder...")
- devices = await BleakScanner.discover(timeout=2.0)
+ devices = await bleak.discover(timeout=2.0)
audio_recorder = None
for device in devices:
if device.name:
print(device.name, device.address)
- if device.address == DEVICE_ID:
+ if device.name == DEVICE_NAME:
audio_recorder = device
break
@@ -34,21 +53,48 @@ async def main():
def handle_ble_disconnect(client):
print("Disconnected from AudioRecorder")
+ # def ulawToLinear(ulawByte):
+ # ulawByte = ~ulawByte & 0xFF # Invert and mask to 8-bit
+ # sign = (ulawByte & 0x80)
+ # exponent = (ulawByte >> 4) & 0x07
+ # mantissa = ulawByte & 0x0F
+ # sample = (0x80 + (mantissa << 4)) << max(0, exponent - 1)
+ # sample -= 0x84 << 2 # Adjusting back the bias used during encoding
+
+ # if sign == 0:
+ # sample = -sample
+
+ # # Adjust output to fit into 16-bit PCM range
+ # return sample & 0xFFFF
+
def filter_audio_data(audio_data):
- audio_data = np.frombuffer(audio_data, dtype=np.uint16)
- audio_data -= 32768
- scaling_factor = 2*32768 / (max(0, np.max(audio_data)) - min(0, np.min(audio_data)))
- return (audio_data * scaling_factor).astype(np.int16)
+
+ if CODEC == "mulaw":
+ # pcm16_samples = audioop.ulaw2lin(audio_data, 2)
+ # pcm16_samples = struct.unpack('<' + 'h' * (len(pcm16_samples) // 2), pcm16_samples)
+ # print(pcm16_samples)
+ pcm16_samples = ulaw_bytes_to_pcm16(audio_data)
+ audio_data = np.array(pcm16_samples, dtype=np.int16)
+
+ if CODEC == "pcm":
+ audio_data = audio_data[:len(audio_data) - len(audio_data) % 2]
+ audio_data = np.frombuffer(audio_data, dtype=np.int16)
+
+ # Normalize
+ # scaling_factor = 2*32768 / (max(0, np.max(audio_data)) - min(0, np.min(audio_data)))
+ # return (audio_data * scaling_factor).astype(np.int16)
+ return audio_data
- def export_audio_data(filtered_audio_data, file_extension):
+ def export_audio_data(filtered_audio_data, raw_file, file_extension):
recordings_dir = "recordings"
if not os.path.exists(recordings_dir):
os.makedirs(recordings_dir)
filename = os.path.join(recordings_dir, datetime.now().strftime("%H-%M-%S-%f") + file_extension)
print(filename)
+ # filename = os.path.join(recordings_dir, "recording" + file_extension)
if file_extension == ".txt":
with open(filename, "w") as file:
- file.write(str(list(filtered_audio_data)))
+ file.write(str(list(raw_file)))
else:
# Directly use the filename with wave.open for .wav files
with wave.open(filename, "wb") as wav_file:
@@ -64,8 +110,8 @@ async def process_audio(audio_data):
return
filtered_audio_data = filter_audio_data(audio_data)
- export_audio_data(filtered_audio_data, ".wav")
- export_audio_data(filtered_audio_data, ".txt")
+ export_audio_data(filtered_audio_data, audio_data, ".wav")
+ export_audio_data(filtered_audio_data, audio_data, ".txt")
pass
async with BleakClient(audio_recorder.address, services=[SERVICE_UUID], disconnect_callback=handle_ble_disconnect) as client:
@@ -78,9 +124,9 @@ async def process_audio(audio_data):
# end_signal = b"\xFF"
def handle_audio_data(sender, data):
- print("---handle_audio_data---")
- print(f"Received {len(data)} bytes at {time.time()}")
- audio_data.extend(data)
+ # print("---handle_audio_data---")
+ # print(f"Received {len(data)} bytes at {time.time()}")
+ audio_data.extend(data[3:])
# if data == [end_signal]:
# print(f"End signal received after {len(audio_data)} bytes")
@@ -95,6 +141,7 @@ async def record_audio():
async def record_and_process():
while True:
await record_audio()
+ print(len(audio_data))
asyncio.ensure_future(process_audio(audio_data.copy()))
audio_data.clear()