Voice messages #4433

stefanceriu · 2021-06-15T14:35:04Z

fix #4090, fix #4091, fix #4092, fix #4094, fix #4095, fix #4096

Added initial voice messages implementation:

custom audio recorder based on AVAudioRecorder
new recording toolbar: long press to record, swipe left to cancel, release to send, duration label and red dot animation
custom audio player based on AVPlayer
audio message bubble cells which still supports all interaction types (reply, react etc.)
custom audio waveformView with sampling done through DSWaveformImage's WaveformAnalyzer
custom audio playback view

Notes:

currently recording and seding MPEG4AAC .m4a files, we need to replace that with opus ogg
missing locked mode recording UI
feature flag unde BuildSettings.voiceMessagesEnabled

- Added voice message button - voice recording UI state

… links. Working audio file sending and cancellation.

…wer levels from audio recorder.

… mode and real time waveform.

…e recording in locked mode.

Riot/Modules/Room/RoomViewController.m

Riot/Modules/Room/Views/InputToolbar/RoomInputToolbarView.m

Riot/Modules/Room/RoomViewController.m

Riot/Modules/Room/VoiceMessages/VoiceMessageController.swift

Riot/Modules/Room/VoiceMessages/VoiceMessageToolbarView.swift

Riot/Utils/WeakObjectWrapper.swift

Riot/Modules/Room/VoiceMessages/VoiceMessageAudioRecorder.swift

…iew.

…ceProvider that prevents simultaneous playback from multiple player instances.

…ion.

…mpling caching layer.

…on by 0.

…ixed completionCallback storage so they take the requestedNumberOfSamples into account.

… integer.

…ue and fixed completionCallback storage so they take the requestedNumberOfSamples into account." This reverts commit 9a3d26e.

…ack to deployment target 11.0.

…sing queue and fixed completionCallback storage so they take the requestedNumberOfSamples into account."" This reverts commit cf884c6.

Riot/Modules/Settings/SettingsViewController.m

Podfile.lock

Riot/Modules/Room/VoiceMessages/VoiceMessageWaveformView.swift

Riot/Modules/Room/VoiceMessages/VoiceMessageToolbarView.swift

Riot/Modules/Room/VoiceMessages/VoiceMessagePlaybackController.swift

Riot/Modules/Room/VoiceMessages/VoiceMessageMediaServiceProvider.swift

Riot/Modules/Room/VoiceMessages/VoiceMessageController.swift

Riot/Modules/Room/VoiceMessages/VoiceMessageAudioRecorder.swift

Riot/Modules/Room/VoiceMessages/VoiceMessageAudioPlayer.swift

pixlwave · 2021-07-19T11:27:11Z

Riot/Modules/Room/VoiceMessages/VoiceMessageAudioRecorder.swift

+
+    func recordWithOuputURL(_ url: URL) {
+
+        let settings = [AVFormatIDKey: Int(kAudioFormatMPEG4AAC),


Probably not an issue, but as an audio nerd it bothers me that this records to a lossy format rather than lossless when it's going to be converted to another lossy format immediately before sending. 🤓

Talked about this privatelly and decided to try to only switch the container format when converting, using opus here.

Riot/Modules/Room/VoiceMessages/VoiceMessageController.swift

… to DateFormatters for formatting durations, sanitising audio player durations and current times.

…back.

MacLemon · 2021-07-22T14:15:51Z

Will there be a possibility for Matrix servers to refuse that message type to conserve storage space?

I'm honestly not keen on receiving any audio from people who are too lazy to type or unwilling to use voice recognition to accommodate for their laziness.

stefanceriu changed the title ~~Element 4090~~ Voice messages Jun 15, 2021

stefanceriu force-pushed the element_4090 branch 2 times, most recently from 3e52d2f to da7a136 Compare June 18, 2021 14:59

gileluard and others added 12 commits June 22, 2021 09:42

Voice Messages - Hold and send

93fd5cd

- Added voice message button - voice recording UI state

#4090 - New voice recording toolbar ui and swipe to cancel interaction.

3624d2e

#4090 - Corrected dark theme appearance.

3015cba

#4090 - Corrected dark theme appearance.

b507097

#4090 - Add voice message controller, audio recorder and toolbar view…

8d54e03

… links. Working audio file sending and cancellation.

#4090 - Added recording duration label and permissions checking.

58c1be0

#4096 - Added voice message decoding, timeline cell and playback UI.

ff3170e

#4090 - Added voice messages feature flag.

1294c27

#4096 - Voice message playback view theme support.

53307f9

#4096 - Various small UI tweaks and bug fixes.

3c1bc56

#4096 - Extracted logic from the VoiceMessagePlaybackView. Exposed po…

99eb2d1

…wer levels from audio recorder.

#4096 - Added locked mode transition and animations, locked recording…

3130fd5

… mode and real time waveform.

stefanceriu force-pushed the element_4090 branch from da7a136 to 3130fd5 Compare June 22, 2021 06:42

stefanceriu added 3 commits June 22, 2021 09:59

#4096 - Weakify display link targets.

02da2e2

#4094 - Added voice messages locked mode playback.

959ccc9

#4094 - Fixed voice message toolbar states after sending message whil…

be8e0f6

…e recording in locked mode.

SBiOSoftWhare reviewed Jun 22, 2021

View reviewed changes

stefanceriu added 3 commits June 23, 2021 15:37

#4094 - Various tweaks, bug fixes and improvements following code rev…

fc7d311

…iew.

#4094 - Added multiple observation on media services and a mediaServi…

7d257f5

…ceProvider that prevents simultaneous playback from multiple player instances.

#4094 - Optimize expensive date formatters creation.

1b90b25

stefanceriu force-pushed the element_4090 branch from 667d590 to 1b90b25 Compare June 24, 2021 08:33

stefanceriu added 6 commits June 24, 2021 12:28

#4094 - Added Opus Ogg support through FFmpegKit backed media convers…

aa60644

…ion.

#4094 - Added toast notifications and maximum recording duration.

c12a7dc

Merge branch 'develop' into element_4090

a807360

#4094 - Added voice message attachment decryption, transcoding and sa…

acebef7

…mpling caching layer.

#4094 - Fixed improper weak display link targets.

24f06a7

#4094 - Caching layer work queue fixes and preventing sampling divisi…

329215d

…on by 0.

stefanceriu requested a review from gaelledel July 14, 2021 12:59

gaelledel approved these changes Jul 15, 2021

View reviewed changes

stefanceriu added 7 commits July 16, 2021 09:06

#4094 - Redrawing waveforms on bound changes.

521c713

#4094 - Reintroduced serial attachment loading processing queue and f…

9a3d26e

…ixed completionCallback storage so they take the requestedNumberOfSamples into account.

#4090 - Switched the sendVoiceMessage method duration parameter to an…

511569a

… integer.

Revert "#4094 - Reintroduced serial attachment loading processing que…

cf884c6

…ue and fixed completionCallback storage so they take the requestedNumberOfSamples into account." This reverts commit 9a3d26e.

#4090 - Changed FFMpeg-Kit to the LTS version and moved the project b…

488338f

…ack to deployment target 11.0.

Revert "Revert "#4094 - Reintroduced serial attachment loading proces…

427e77e

…sing queue and fixed completionCallback storage so they take the requestedNumberOfSamples into account."" This reverts commit cf884c6.

#4090 - Fixed the AttachmentCacheManager's serial nature.

ec33ec8

stefanceriu force-pushed the element_4090 branch from 79bb35d to ec33ec8 Compare July 16, 2021 12:48

#4090 - Added voice messages switch to the labs section in settings.

089c688

pixlwave reviewed Jul 16, 2021

View reviewed changes

Riot/Modules/Settings/SettingsViewController.m Outdated Show resolved Hide resolved

pixlwave reviewed Jul 16, 2021

View reviewed changes

Podfile.lock Show resolved Hide resolved