Skip to content

Commit

Permalink
added documentation about audio decoding
Browse files Browse the repository at this point in the history
  • Loading branch information
@abudaan
abudaan committed Jan 3, 2015
1 parent b09d1e6 commit 5fa21f0
Show file tree
Hide file tree
Showing 5 changed files with 130 additions and 3 deletions.
84 changes: 84 additions & 0 deletions docs/decoding-audio/index.html
View file
@@ -0,0 +1,84 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>heartbeat</title>
<meta name="viewport" content="width=device-width,initial-scale=1.0,maximum-scale=1.0,user-scalable=no">
<link rel="stylesheet" type="text/css" href="/heartbeat/fonts/fonts.css">
<link rel="stylesheet" type="text/css" href="/heartbeat/css/reset.css">
<link rel="stylesheet" type="text/css" href="/heartbeat/css/prism.css">
<link rel="stylesheet" type="text/css" href="/heartbeat/css/docs.css">
<script src="/heartbeat/js/open.js"></script>
<script src="/heartbeat/js/create_navigation.js"></script>
<script src="/heartbeat/js/create_tabs.js"></script>
<script src="/heartbeat/js/navigation_items.js"></script>
<script src="/heartbeat/js/util.js"></script>
<script src="/heartbeat/js/close.js"></script>
<script src="/heartbeat/js/main.js"></script>
<script>
if(window.location.hostname === 'abudaan.github.io'){
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');

ga('create', 'UA-5081547-9', 'heartbeatjs.org');
ga('require', 'displayfeatures');
ga('require', 'linkid', 'linkid.js');
ga('send', 'pageview');
}
</script>
</head>
<body>
<div id='tab_buttons'>
<ul>
<li id='tab_close' class='tab_active'>&#8593;</li>
<li id='tab_properties' class='tab_active'>properties</li>
<li id='tab_methods' class='tab_active'>methods</li>
</ul>
</div>
<div id='tab_content'>
<div id='tab_column_container'></div>
</div>
<div id="main">
<div id="content">
<h1>Decoding audio</h1>

<p><a name="Background"></a></p>

<h3>Background</h3>

<p>Before you can play audio files via the <a href="https://developer.mozilla.org/en-US/docs/Web/API/AudioContext">AudioContext</a>, the audio files need to be parsed to the format that the AudioContext uses for internal processing. This format is non-interleaved 32-bit linear PCM. This means that every audio sample is a float value between -1.0 and 1.0 and that the samples are stored per channel.</p>

<p>Linear PCM, or LPCM is a form of PCM sampling, more information can be found on <a href="http://en.wikipedia.org/wiki/Pulse-code_modulation">this wikipedia page</a>.</p>

<p>You can convert an audio file to the required format by using <code class="language-javascript">WebAudioContext.decodeAudioBuffer(buffer)</code>. The buffer argument is the audio file in <a href="https://developer.mozilla.org/en-US/search?q=arraybuffer">ArrayBuffer</a> format. If you load the audio file via an XMLHttpRequest you have to set the response type to <code class="language-javascript">arraybuffer</code>. The decoding process returns an instance of <a href="https://developer.mozilla.org/en-US/docs/Web/API/AudioBuffer">AudioBuffer</a>.</p>

<p>An AudioBuffer stores the samples per channel in an <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Float32Array">Float32Array</a>. You can easily access the ArrayBuffers of a specific channel with <code class="language-javascript">AudioBuffer.getChannelData(index)</code>. If your file is mono, the maximum channel index is 0. If your file is stereo channel index 0 will retrieve the samples of the left channel and channel index 1 will retrieve the samples of the right channel.</p>

<p><a name="Compressed-audio-vs-uncompressed-audio"></a></p>

<h3>Compressed audio vs uncompressed audio</h3>

<p>A 16-bit wav file stores the audio samples as interleaved 32-bit PCM samples. Interleaved means that the first sample is the first sample of the left channel, the second sample is the first sample of the right channel. Then the third sample is the second sample of the left channel and the fourth sample is second sample of the right channel, and so on. The samples are stored as values between -32,768 and 32,767.</p>

<p>Converting a wav file to the format that the AudioContext requires is quite straightforward; the samples need to be stored in a separate AudioBuffer for each channel, and the value of each sample needs to be remapped to a value between -1.0 and 1.0.</p>

<p>Since the data type of a sample and the number of samples in a wav files is exact the same as in an AudioBuffer, the file size of a wav file is about the same as the memory used by an AudioBuffer once it has been loaded and decoded in your browser.</p>

<p>Converting compressed formats like mp3 and ogg is a bit more complex; whereas a wav file already contains every sample in almost the required format, for mp3 and other compressed formats every sample literally needs to be decoded. For more information about decoding mp3 see <a href="http://blog.bjrn.se/2008/10/lets-build-mp3-decoder.html">this blog</a>.</p>

<p>Therefor decoding compressed formats takes much longer than decoding uncompressed formats. In fact when you decode a wav file, no actual decoding takes place because a wav file actually is a slightly different organized kind of AudioBuffer.</p>

<p>In this <a href="/heartbeat/examples/#!decode_audio_test">example</a> you can see how long it takes to decode a file in wav, ogg and mp3 format. As expected, wav decodes the fastest, then ogg, then mp3. Note that the time it takes to decode a compressed file is also dependent on the used encoder and the encoding settings.</p>

<p>You can find more information about audio and the AudioContext on <a href="https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Basic_concepts_behind_Web_Audio_API">this MDN page</a>.</p>
</div><!-- end of div container-->
</div><!-- end of div main-->
<div id="side">
<div id="logo">HEART<span id="logo_beat">BEAT</span><span id="logo_js">JS</span></div>
<nav></nav>
</div>
<script src="/heartbeat/js/prism.js"></script>
</body>
</html>
8 changes: 6 additions & 2 deletions examples/decode_audio_test/index.html
View file
Expand Up @@ -15,15 +15,19 @@
</head>
<body>
<p>
This example shows that decoding a wav file is much faster than decoding a compressed format like mp3. Both the short track (8 seconds) and the long track (6 minutes and 25 seconds) are decoded about 4 times faster in uncompressed format.
This example shows that decoding a wav file is much faster than decoding a compressed format like ogg or mp3. Both the short track (8 seconds) and the long track (6 minutes and 25 seconds) are decoded a couple of times faster in uncompressed format. Note that decoding ogg is faster than decoding mp3.
</p>
<p>
Obviously, loading the wav files takes much longer. But if your application loads audio files locally (for instance if your application is a mobile app that uses persistent local storage) using wav files might be preferable.
</p>
<p>
The long track in wav form is about 68 MB, the short track is 1.4 MB. The mp3 versions are respectively 6.2 MB and 193 KB in file size. The loading and decoding durations are printed to the console of your browser.
The long track in wav format is about 68 MB, the short track is 1.4 MB. The mp3 versions are respectively 6.2 MB and 193 KB in file size, the ogg versions resp. 5.5 MB and 103K. The loading and decoding durations are printed to the console of your browser.
</p>
<p>
More documentation about audio decoding can be found <a target="_parent" href="/heartbeat/docs/decoding-audio/">here</a>
</p>
<input type="button" id="wav" value="wav">
<input type="button" id="ogg" value="ogg">
<input type="button" id="mp3" value="mp3">
<input type="button" id="stop" value="stop">
<select id="select_track">
Expand Down
3 changes: 3 additions & 0 deletions examples/decode_audio_test/main.css
View file
Expand Up @@ -6,4 +6,7 @@ ul{
list-style-type: disc;
list-style-position: inside;
margin-bottom: 20px;
}
p{
margin-bottom: 15px;
}
35 changes: 35 additions & 0 deletions examples/decode_audio_test/main.js
View file
Expand Up @@ -8,6 +8,7 @@ window.onload = function(){
console = window.console,

btnWav = document.getElementById('wav'),
btnOgg = document.getElementById('ogg'),
btnMp3 = document.getElementById('mp3'),
btnStop = document.getElementById('stop'),
divConsole = document.getElementById('console'),
Expand Down Expand Up @@ -68,6 +69,36 @@ window.onload = function(){
}, false);


btnOgg.addEventListener('click', function(){
if(src !== undefined){
src.stop();
}
divConsole.innerHTML = 'loading ogg...';
console.time('loading ogg took');
sequencer.util.ajax({
url: track + '.ogg',
responseType: 'arraybuffer',
onError: function(e){
console.log(e);
},
onSuccess: function(buffer){
console.timeEnd('loading ogg took');
divConsole.innerHTML = '';
setTimeout(function(){
console.time('decoding ogg took');
context.decodeAudioData(buffer, function(buffer){
console.timeEnd('decoding ogg took');
src = context.createBufferSource();
src.buffer = buffer;
src.connect(context.destination);
src.start();
});
},0);
}
});
}, false);


btnMp3.addEventListener('click', function(){
if(src !== undefined){
src.stop();
Expand Down Expand Up @@ -104,6 +135,9 @@ window.onload = function(){


enableUI(true);

btnOgg.disabled = sequencer.ogg === false;
btnMp3.disabled = sequencer.mp3 === false;
});


Expand All @@ -115,5 +149,6 @@ window.onload = function(){
element = elements[i];
element.disabled = !flag;
}

}
};
3 changes: 2 additions & 1 deletion js/navigation_items.js
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 5fa21f0

Please sign in to comment.