Latest development version of Firefox supports HTML5 playback of mp3 and mp4 using gstreamer. Still probably requires custom building of Firefox, the Mozilla tracking bug is here.
Using mp3 and mp4 gstreamer packages could expose you to royalty claims from the patent owners.
Gstreamer is newly integrated in Archlinux with Firefox. I'm looking to move to Archlinux.
http://www.mozilla.org/en-US/firefox/26.0/releasenotes/ seems to imply better Gstreamer integration. Need to test it.
Finally managed to get it working. But it brings a crap ton of dependencies in:
dbus-x11 dconf-gsettings-backend dconf-service gconf-service gconf2 gconf2-common glib-networking glib-networking-common
glib-networking-services gsettings-desktop-schemas gstreamer0.10-doc gstreamer0.10-ffmpeg gstreamer0.10-gconf
gstreamer0.10-plugins-base gstreamer0.10-plugins-good gstreamer0.10-pulseaudio gstreamer0.10-x iso-codes libaa1 libavc1394-0
libcaca0 libcairo-gobject2 libcdparanoia0 libdconf0 libdv4 libgconf-2-4 libglib2.0-doc libgnome-keyring-common libgnome-keyring0
libgpm2 libgstreamer-plugins-base0.10-0 libgstreamer0.10-0 libgudev-1.0-0 libiec61883-0 libpostproc52 libproxy0 libraw1394-11
libshout3 libsoup-gnome2.4-1 libsoup2.4-1 libsqlite3-0 libtag1-vanilla libtag1c2a libvisual-0.4-0 libwavpack1 libxml2-doc
mime-support psmisc python python-minimal python2.7 python2.7-minimal
Furthermore for MP3
liba52-0.7.4 libcdio13 libdvdread4 libmad0 libmpeg2-4 libopencore-amrnb0 libopencore-amrwb0 libsidplay1 libtwolame0 gstreamer0.10-plugins-ugly
All in all, making the final image about 20M bigger in size and taking factors of that more in RAM when uncompressed.
gstreamer0.10-plugins-good seems like a particularly naughty package.
Support MP4 HTML video playback, fixes #161
Support MP3 HTML audio playback, fixes #161