Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Story] Receive and play a voice message #2083

Closed
18 of 23 tasks
Tracked by #2057
jonnyandrew opened this issue Sep 29, 2023 · 2 comments
Closed
18 of 23 tasks
Tracked by #2057

[Story] Receive and play a voice message #2083

jonnyandrew opened this issue Sep 29, 2023 · 2 comments
Assignees
@nimau
Labels
App: ElementX Android App: ElementX iOS T-User Story Team: Element X Feature X-Needs-Signoff Stories and Epics which are ready for review by product, design and QA

Comments

@jonnyandrew
Copy link

jonnyandrew commented Sep 29, 2023 •
edited by julioromano

Description

Receive and play voice messages in the timeline.

Acceptance criteria

  • It is behind a feature flag in developer settings
  • It displays in the timeline
  • It can be played from the timeline
  • It works in encrypted and unencrypted rooms
  • Message continues playing after scrolling out of view
  • Navigating away from the room stops playback
  • Play and pause the voice message
  • Display length of message and play/pause state
  • It is compatible with MSC3245 v1
  • Sending an audio file attachment continues to work as expected
  • It interoperates with classic Element mobile and web apps
  • Files are not exposed through system file managers
  • It works in encrypted and unencrypted rooms
  • If you delete a voice message while it is playing it should stop playing
  • Playing multiple messages one after the other should continue to work as expected.
  • While voice message is a local echo in the timeline, the play button is disabled.

Size estimate

None

Dependencies

No response

Preconditions

No response

Sign-off

Android

  • Design sign-off on completion
  • QA sign-off on completion
  • Product sign-off on completion

iOS

  • Design sign-off on completion
  • QA sign-off on completion
  • Product sign-off on completion

Out of scope

  • Preview the voice message before sending

Open questions

Subtasks

Android
Beta Give feedback
  6. Receive and play a voice message vector-im/element-x-android#1503
    julioromano
  7. Add global context.cacheDir provider. vector-im/element-x-android#1606
    julioromano
  8. Extract more content from audio messages. vector-im/element-x-android#1607
    julioromano
  9. TimelineItemPresenterFactories vector-im/element-x-android#1609
    julioromano
  10. Voice message bubble design improvements vector-im/element-x-android#1637
    julioromano
  11. Refactor of voice message playing code vector-im/element-x-android#1643
    julioromano
  12. Move MediaPlayer from messages module to new dedicated library module vector-im/element-x-android#1654
    julioromano
  13. Add missing screenshots in TimelineView vector-im/element-x-android#1660
    julioromano
  14. Split VoiceMessageType from AudioMessageType vector-im/element-x-android#1664
    julioromano
  15. Enable swipe to reply on voice messages vector-im/element-x-android#1668
    julioromano
  16. Voice message button: Show proper disabled color. vector-im/element-x-android#1682
    julioromano
  17. Add a11y label to voice messages vector-im/element-x-android#1683
    julioromano
  18. Stop voice message on redaction vector-im/element-x-android#1826
    julioromano

iOS
Beta Give feedback

Other
Beta Give feedback
No tasks being tracked yet.
This was referenced Sep 29, 2023
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
@julioromano julioromano self-assigned this Oct 6, 2023
@nimau nimau self-assigned this Oct 6, 2023
@jonnyandrew jonnyandrew mentioned this issue Oct 6, 2023
Merged
14 tasks
@langleyd
Copy link
Contributor

Added a user case in the acceptance criteria regarding this bug on EI -> element-hq/element-ios#7190

@langleyd
Copy link
Contributor

Added a use case in the acceptance criteria for playing multiple voice messages(one after the other) based on a bug in EA -> element-hq/element-android#8412

@julioromano julioromano mentioned this issue Oct 21, 2023
Merged
4 tasks
julioromano added a commit to element-hq/element-x-android that referenced this issue Oct 24, 2023
@julioromano
Receive and play a voice message (#1503)
6e66c98 
## Type of change

- [x] Feature
- [ ] Bugfix
- [ ] Technical
- [ ] Other :

## Content

This PR consists of several macro-blocks separated by path/package:
- `messages.impl.mediaplayer` : Global (room-wide) media player, now used only for voice messages but could be used for all media within EX in the future. It is backed by media3's exoplayer. Currently not unit-tested because mocking exoplayer is not trivial.
- `messages.impl.voicemessages.play` : Business logic of a timeline voice message. This is all the logic that manages the voice message bubble.
- `messages.impl.timeline.model` & `messages.impl.timeline.factories`: Timeline code that takes care of creating the `content` object for voice messages.
-  `messages.impl.timeline.components` : The actual View composable that shows the UI inside a voice message bubble.

All the rest is just small related changes that must be done here and there in existing code.

From a high level perspective this is how it works:
- Voice messages are unlike other message bubbles because they carry state (i.e. playing, downloading...) so they have a Presenter managing this state.
- Media content (i.e. the ogg file) of a voice message is downloaded from the rust SDK on first play then stored in a voice messages cache (see the `VoiceMessageCache` class, it is just a subdirectory in the app's cacheDir which is indexed by the matrix content uri). All further play attempts are done from the cache without hitting the rust SDK anymore.
- Playback of the ogg file is handled with the `VoiceMessagePlayer` class which is basically a "view" of the global `MediaPlayer` that allow the voice message to only see the media player state belonging to its media content. 
- Drawing of the waveform is done with an OSS library wrapped in the `WaveformProgressIndicator` composable.

Known issues:
 - The waveform has no position slider.
 - The waveform (and together with it the whole message bubble) is taller than the actual Figma design.
 - Swipe to reply for voice messages is disabled to avoid conflict with the audio scrubbing gesture (to reply to a voice message you have to use the long press menu).
 - The loading indicator is always shown (there is no delay).
 - Voice messages don't stop playing when redacted.

## Motivation and context

element-hq/element-meta#2083

## Screenshots / GIFs

Provided by Screenshot tests in the PR itself.
@langleyd langleyd added the X-Needs-Signoff Stories and Epics which are ready for review by product, design and QA label Nov 9, 2023
@julioromano julioromano removed their assignment Dec 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nimau nimau

Labels
App: ElementX Android App: ElementX iOS T-User Story Team: Element X Feature X-Needs-Signoff Stories and Epics which are ready for review by product, design and QA
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@julioromano @nimau @jonnyandrew @langleyd @VolkerJunginger