Record audio button #113

EloxZ · 2023-09-18T18:07:16Z

Added a button in freeplay TopBar to record audio
I'm using a recordingDestinationNode that listens every gainNode and is recorded with MediaRecorder
Saving file in webm to support most browsers (Tested in Edge and Firefox)

EloxZ · 2023-09-18T19:57:27Z

TODO:

Webm files doesn't support seeking, convert to wav or mp3
Save a MIDI file too
Create a window to decide location/file name
Load a MIDI and share it (my idea is to convert the midi file into a token to be used as parameter, but this token would be too long)

samouri · 2023-09-18T23:44:37Z

Load a MIDI and share it (my idea is to convert the midi file into a token to be used as parameter, but this token would be too long)

You can base64 encode the MIDI and stick it in a query param. It's fine if it is too long, since it'll only be used client side (servers never need to process it).

samouri

I played with the functionality and it worked well! Thank you for the contributions so far. Excited to see what you do with the "save as" dialog 😄

If you'd like, feel free to install and use the Dialog component from Radix-UI

samouri · 2023-09-19T02:53:27Z

src/features/audioRecording/index.ts

+    if (isAble) {
+        const recordingDestinationNode = audioContext.createMediaStreamDestination();
+        setRecordingDestinationNode(recordingDestinationNode);
+        mediaRecorder = new MediaRecorder(recordingDestinationNode.stream, {mimeType: 'audio/webm'});


Can we default to mp3 instead of webm? I think most people using the feature would be surprised to have webm as the default

I've tried but the file somehow comes corrupted. I guess we have to convert it.

src/features/audioRecording/index.ts

src/features/pages/Freeplay/components/TopBar.tsx

src/features/synth/utils.ts

EloxZ · 2023-09-19T17:10:33Z

I've made the midi recording (it only records midi message events, not pressing notes with mouse)

EloxZ · 2023-09-19T20:39:26Z

src/features/pages/PlaySong/index.tsx

useBase64Song() is called endlessly, not sure how to solve it. But the Midi file has been rebuilt properly.

Solved now by updating useSong to understand base64 as a source type

src/features/midi/utils.tsx

samouri

@EloxZ, thank you for the updates! I didn't know about @tonejs/midi, but that seems like it'll be very useful for sightread.

In fact, I think it'd probably be better to replace the implementation of parsers/parseMidi to use @tonejs/midi instead of jasmid since it so much higher level.

Separately, is it possible you forgot to push your latest updates? I don't see any code right now that adds to the recorededMidiEvents array. Testing this resulting in an empty list

Let me know if there's anything you need help with to get this to the finish line. Some things I think it needs before it's ready:

Remove all the console.log statements and unused code / some mild code cleanup
It needs to function correctly
Ideally a prompt to ask if you'd like to copy as a link or download as a webm/mp3, and where to save it to.

I can work on the parseMidi migration out of band

src/features/pages/Freeplay/components/TopBar.tsx

src/features/pages/PlaySong/index.tsx

src/features/midi/index.tsx

src/features/midi/utils.tsx

samouri · 2023-10-15T15:38:54Z

@EloxZ, let me know if you'd like to continue working on this. Otherwise I'll pick it up next week

EloxZ · 2023-11-11T22:18:38Z

@EloxZ, let me know if you'd like to continue working on this. Otherwise I'll pick it up next week

I can continue. Did you do something?

samouri · 2023-11-12T19:03:15Z

@EloxZ, let me know if you'd like to continue working on this. Otherwise I'll pick it up next week

I can continue. Did you do something?

Not yet! I've been busy with other projects

Added button for recording audio in freeplay, it downloads a .webm file for broad browser support

Fix format

* Small bugfixes to the Player. Restarts song when at the end and hitting play. * Removes support for downloading mp3-style audio. MIDI is enough for this PR, future one can use lamemp3 etc to convert the midi to mp3.

samouri · 2024-01-15T01:48:04Z

@EloxZ I've made some updates, and plan to merge it now.
After completing the recording, we now display a "Recording Preview Modal".

I've also removed the webm recording because the implementation was particularly stateful and jankey (bad enough to do it just for MIDI). An ideal implementation should be able to take the recorded midi and deterministically generate an mp3 from it, (e.g. using lamemp3 or mscore) instead of using the Web Audio API's recording capabilities.

Thank you again for kicking this off 😄

samouri reviewed Sep 19, 2023

View reviewed changes

EloxZ commented Sep 19, 2023

View reviewed changes

src/features/midi/utils.tsx Outdated Show resolved Hide resolved

samouri reviewed Sep 23, 2023

View reviewed changes

src/features/pages/Freeplay/components/TopBar.tsx Show resolved Hide resolved

src/features/pages/PlaySong/index.tsx Outdated Show resolved Hide resolved

EloxZ commented Sep 24, 2023

View reviewed changes

src/features/midi/index.tsx Outdated Show resolved Hide resolved

samouri reviewed Sep 24, 2023

View reviewed changes

src/features/midi/utils.tsx Outdated Show resolved Hide resolved

samouri self-assigned this Oct 21, 2023

EloxZ added 5 commits January 14, 2024 20:17

Recording audio working

580a47d

Added button for recording audio in freeplay, it downloads a .webm file for broad browser support

Update index.tsx

20788d1

Update getSynth.ts

bca3c58

Update index.ts

a389f9d

Fix format

Record and load Midi in progress

55e9fff

samouri force-pushed the record-audio-button branch from ce71415 to 55e9fff Compare January 15, 2024 01:19

Sharable URLs, Download MIDI, and Preview Modal

cd72997

* Small bugfixes to the Player. Restarts song when at the end and hitting play. * Removes support for downloading mp3-style audio. MIDI is enough for this PR, future one can use lamemp3 etc to convert the midi to mp3.

samouri force-pushed the record-audio-button branch from fad2caf to cd72997 Compare January 15, 2024 01:25

samouri added 2 commits January 14, 2024 20:34

Lint/TS fixes

6093146

Run prettier

0787d27

samouri force-pushed the record-audio-button branch from 7aab9f1 to 0787d27 Compare January 15, 2024 01:38

samouri added 2 commits January 14, 2024 20:39

Delete unused file

a940276

Update text

a50b401

samouri approved these changes Jan 15, 2024

View reviewed changes

samouri merged commit 59e0506 into sightread:main Jan 15, 2024
1 check passed

samouri mentioned this pull request Jan 15, 2024

Download recording as MP3 #128

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Record audio button #113

Record audio button #113

EloxZ commented Sep 18, 2023

EloxZ commented Sep 18, 2023

samouri commented Sep 18, 2023

samouri left a comment •

edited

samouri Sep 19, 2023

EloxZ Sep 19, 2023

EloxZ commented Sep 19, 2023

EloxZ Sep 19, 2023

samouri Jan 15, 2024 •

edited

samouri left a comment •

edited

samouri commented Oct 15, 2023

EloxZ commented Nov 11, 2023

samouri commented Nov 12, 2023

samouri commented Jan 15, 2024 •

edited

Record audio button #113

Record audio button #113

Conversation

EloxZ commented Sep 18, 2023

EloxZ commented Sep 18, 2023

samouri commented Sep 18, 2023

samouri left a comment • edited

Choose a reason for hiding this comment

samouri Sep 19, 2023

Choose a reason for hiding this comment

EloxZ Sep 19, 2023

Choose a reason for hiding this comment

EloxZ commented Sep 19, 2023

EloxZ Sep 19, 2023

Choose a reason for hiding this comment

samouri Jan 15, 2024 • edited

Choose a reason for hiding this comment

samouri left a comment • edited

Choose a reason for hiding this comment

samouri commented Oct 15, 2023

EloxZ commented Nov 11, 2023

samouri commented Nov 12, 2023

samouri commented Jan 15, 2024 • edited

samouri left a comment •

edited

samouri Jan 15, 2024 •

edited

samouri left a comment •

edited

samouri commented Jan 15, 2024 •

edited