Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add packages/karaoke #65

Merged
merged 17 commits into from
Jul 19, 2022
Merged

Conversation

nickhsine
Copy link
Contributor

@nickhsine nickhsine commented Jul 12, 2022

Feature 描述

根據 mockup 實作卡拉 OK 元件。

互動需求說明:

  1. 當元件進入 viewport 後,字卡和音檔「從頭」自動播放。
  2. 當元件離開 viewport 後,音檔暫停播放。
  3. 若使用者點擊過按鈕,記錄音檔的靜音狀態,當元件再次進入 viewport 時,音檔自動播放時,會透過靜音狀態來決定是否靜音。
  4. 因為瀏覽器的規範,使用者尚未點擊播放按鈕前,音檔會處於靜音模式,當使用者點擊播放按鈕後,音檔會根據字卡的狀態來決定要從哪裡開始播放。

Demo:
karaoke

實作細節

有兩點特殊之處需要先加以說明:

  1. React 的 <audio> tag 不支援 muted 屬性,這是一個 known issue,相關討論請見 <video /> attribute needed but not guaranteed by React facebook/react#10389
  2. 瀏覽器要 autoplay 的話,必須搭配靜音模式 muted={true} 才會生效

原本的實作方式是想透過 <audio autoPlay muted> 來讓瀏覽器幫忙自動播放音檔,並且記錄音檔的播放到哪裡 currentTime。但因為 muted 屬性 React 不能使用,因此,放棄該實作方式,而改用較為複雜的方法。

較為複雜的方法是:

  1. 我們得自己紀錄音檔的 currentTime
  2. 我們得透過 DOM element 來播放 audio.play(),暫停 audio.pause() 和開啟/靜音模式audio.muted=(true|false)

currentTime 的記錄更新是透過 setTimeout 實作,讓字卡一邊跑動畫,一邊更新 currentTime
另外,此實作利用 intersection observer 來判斷元件是否進入 viewport。

因為有可能有多個卡拉 OK 元件出現在同一個網頁上,所以將 muted state 記錄在 window level;當使用者在其中一個卡拉 OK 元件中開啟靜音模式,其他的卡拉 OK 元件也都會一起開啟靜音模式。

TODOs

  • mobile, tablet version
  • add background-image

…manually

We set `audio.currentTime` and `audio.muted` manually for the following situation:
The default behavior is to autoplay the audio and quote shadow animation.

However, the modern browser will block autoplay if the audio
is not set to `muted`. (For more information, please see
https://developer.mozilla.org/en-US/docs/Web/Media/Autoplay_guide#autoplay_availability)
(aftermath: `audio.play()` will return promise rejection)

And the unlucky part is that we cannot set `muted` in the `audio` tag
due to this known issue (facebook/react#10389).
(aftermath: we have to change `muted` attribute by manipulating DOM
element)

Because of the above issues, we cannot play the audio with sound only if
users click the audio button. At the same time, the quote animation works
well.
(aftermath: audio and shadow animation are not synced)

When the users click the audio button, we have to play the audio at the
right `currentTime` to catch up the shadow animation.
(counting the `currentTime` by ourselves)

And this patch handles the above situation.
@nickhsine
Copy link
Contributor Author

@caesarWHLee 有空請幫忙 review 一下。

See related issue: video-dev/hls.js#1686
Since there is a problem with `canplay` event, this patch
removes `audioOpts.canPlay` state to avoid broken functionaility
on sarafi.
@nickhsine nickhsine force-pushed the audio-quote-shadow branch 3 times, most recently from 333fb9d to 6d14ec9 Compare July 14, 2022 02:39
… click the buttons

This patch fixes a corner case.
On safari, `audio.play()` still returns Promise rejection even users
have already had interactions (clicked the buttons).

My guess is because `audio` DOM element is regenerated due to
`Karaoke` component's re-rendering.

This patch moves `<audio>` tag out of `<Container>`.
Therefore, even if `Karaoke` component re-renders, React won't
regenerate a new `audio` DOM element.
@dyfu95
Copy link
Collaborator

dyfu95 commented Jul 14, 2022

Question:
我自己測試的結果,目前會有同時出現兩個音軌同時播放的情形,不確定是否需要解掉這狀況?
如下圖:當音軌一與音軌二同時在viewport中,就會同時播放
Screen Shot 2022-07-14 at 12 28 23 PM

commit 3c10d01 only fixes one Karaoke
component on the Safari browser.

If there are several Karaoke components on the Safari browser,
`audio.play()` still encounter Promise rejection.
@nickhsine
Copy link
Contributor Author

3c10d01 and 06eb881 修復了 Safari 在自動播放上的問題。
該問題只出現在 Sarafi 瀏覽器上。
假設我們有三個 Karaoke 元件在同一頁上,當使用者點擊其中一個 Karaoke 元件 A 的播放鍵後,Safari 確實可以讓被點擊的 Karaoke 元件 A 播放聲音,但當使用者往下滑,進入另一個 Karaoke 元件 B 後,Karaoke 元件 B 在執行 audio.play() 時,仍會遇到 Promise rejection 的問題。
06eb881 透過 workaround 的方式解到該問題。

@nickhsine
Copy link
Contributor Author

Question: 我自己測試的結果,目前會有同時出現兩個音軌同時播放的情形,不確定是否需要解掉這狀況? 如下圖:當音軌一與音軌二同時在viewport中,就會同時播放

這個我會再調整 intersection observer 的 threshold 試試看。

packages/karaoke/README.md Outdated Show resolved Hide resolved
packages/karaoke/dev/entry.js Outdated Show resolved Hide resolved
packages/karaoke/dev/entry.js Outdated Show resolved Hide resolved
packages/karaoke/src/react-components/index.js Outdated Show resolved Hide resolved
packages/karaoke/src/react-components/quote-shadow.js Outdated Show resolved Hide resolved
nickhsine added a commit to nickhsine/readr-react that referenced this pull request Jul 14, 2022
nickhsine added a commit to nickhsine/readr-react that referenced this pull request Jul 14, 2022
nickhsine added a commit to nickhsine/readr-react that referenced this pull request Jul 14, 2022
@nickhsine
Copy link
Contributor Author

nickhsine commented Jul 14, 2022

  • 09b4902 更新 QuoteShadow 中的 useEffect 使用方式,帶入 array 避免 re-render 時 useEffect 重跑
  • 92bae9e 拿掉多餘的 React.Fragment
  • 238d952[...audioUrls] 改成 []
  • e4510bf 修掉其它 review comments

@dyfu95
Copy link
Collaborator

dyfu95 commented Jul 15, 2022

LGTM 👍 👍

@caesarWHLee
Copy link
Collaborator

LGTM 💯 ⭐ ✨

@nickhsine
Copy link
Contributor Author

@caesarWHLee @dyfu95
關於 #65 (comment) commit thread,我打算用另外一個 PR 來優化 useEffect
我先 merge 這個 PR 囉。

@nickhsine nickhsine merged commit 03df3a9 into readr-media:main Jul 19, 2022
@nickhsine nickhsine deleted the audio-quote-shadow branch December 6, 2023 05:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants