Browse files

Merge remote-tracking branch 'origin/development'

  • Loading branch information...
2 parents ca7e599 + 3481192 commit 23b20d2e8fca1ab7f0accd53943978e762c5e82b @jbillo committed Sep 3, 2012
Showing with 210 additions and 44 deletions.
  1. +98 −6 README.md
  2. +1 −1 decoder.py
  3. +31 −2 file_utils.py
  4. +1 −1 reference_frame.py
  5. +79 −34 xenonmkv.py
View
104 README.md
@@ -6,7 +6,7 @@ You'll find this tool useful to converting videos for devices that support H.264
# System Requirements
-XenonMKV was built and tested on a standard Ubuntu 12.04 LTS installation, but most of the utilities and requirements here are available for most popular *nix distributions.
+XenonMKV was built and tested on a standard Ubuntu 12.04 LTS installation (x86_64), but most of the utilities and requirements here are available for most popular *nix distributions. You will need at least Python 2.7 for the argparse library, and all code was tested on Python 2.7.3.
## Packages to Install
@@ -19,31 +19,58 @@ You will need some supporting packages. Instructions below work on Ubuntu 12.04.
### Individual Package Requirements
* mediainfo
- http://mediainfo.sourceforge.net/en/Download/Ubuntu
+ <http://mediainfo.sourceforge.net/en/Download/Ubuntu>
+
+ sudo apt-get install mediainfo
+
+ Alternatively, add the official mediainfo PPA and install the package, which the developer suggests:
+
+ sudo add-apt-repository ppa:shiki/mediainfo
+ sudo apt-get update
sudo apt-get install mediainfo
* mkvtoolnix
- http://www.bunkus.org/videotools/mkvtoolnix/downloads.html
+
+ <http://www.bunkus.org/videotools/mkvtoolnix/downloads.html>
sudo apt-get install mkvtoolnix
* mplayer
- http://www.mplayerhq.hu/design7/news.html
+
+ <http://www.mplayerhq.hu/design7/news.html>
sudo apt-get install mplayer
* faac
- http://www.audiocoding.com/downloads.html
+
+ <http://www.audiocoding.com/downloads.html>
sudo apt-get install faac
* MP4Box
- https://sourceforge.net/projects/gpac/
+
+ <https://sourceforge.net/projects/gpac/>
sudo apt-get install gpac
+## Ubuntu 10.04
+
+As I still have a few systems around running Ubuntu 10.04, here are the changes required to make XenonMKV functional:
+
+* Install Python 2.7, either from source or add the appropriate PPA:
+
+ sudo add-apt-repository ppa:fkrull/deadsnakes
+ sudo apt-get update
+ sudo apt-get install python2.7
+
+* Install mediainfo by adding the PPA referenced above as it is not in the 10.04 package repository.
+
+* Run the application directly referencing Python 2.7:
+
+ /usr/bin/python2.7 /path/to/xenonmkv.py [arguments]
+
# Suggested Applications
* vlc
@@ -56,6 +83,10 @@ You will need some supporting packages. Instructions below work on Ubuntu 12.04.
Basic usage with default settings:
xenonmkv.py /path/to/file.mkv
+
+To ensure your Xbox 360 will play the resulting file, at a possible expense of audio quality:
+
+ xenonmkv.py /path/to/file.mkv --profile xbox360
To see all command line arguments:
@@ -80,5 +111,66 @@ For a quiet run (batch processing or in a cronjob):
You could probably get much better performance with a solid state drive, and obviously processor speed will have an impact here.
+# Audio Downmixing/Re-Encoding
+
+By default, XenonMKV tries not to have to resample, downmix or re-encode any part of the content provided. However, chances are your source files will contain AC3, DTS or MP3 audio that needs to be re-encoded. In this case, the original source audio will always be downmixed to a two channel AAC file before it is repackaged.
+
+If the audio track in your MKV file is already AAC, the next thing to consider is your playback device. The Xbox 360 will not play audio in an MP4 container unless it is 2-channel stereo, which is a highly stupid limitation. Other devices, like the PlayBook, will happily parse up to 5.1 channel audio. By using either the "--channels" or "--profile" settings, you can tell XenonMKV how many channels of audio are acceptable from an AAC source before it will aggressively re-encode and downmix to 2-channel stereo.
+
+In short, if you plan to play MP4s on your Xbox 360, definitely use the "--profile xbox360" setting to make sure that no more than two channels make it into the output file. If your device is more reasonable, the default settings should be fine. More profiles will be added as users confirm their own device capabilities.
+
+# Known Issues
+
+## MP4Box crash with backtrace
+
+Certain video files, when MP4Box loads them to rejoin into an MP4 container, will throw a glibc error beginning with:
+
+ *** glibc detected *** MP4Box: free(): invalid next size (fast): 0x0000000000cc8400 ***
+ ======= Backtrace: =========
+ /lib/x86_64-linux-gnu/libc.so.6(+0x7e626)[0x7f0b09a77626]
+ /usr/lib/nvidia-current/tls/libnvidia-tls.so.295.40(+0x1c01)[0x7f0b084c7c01]
+
+This occurs with both nVidia proprietary and Nouveau open source drivers. The message above is displayed when using the "current" or "current-updates" versions (295.40, 295.49). When using the 173, 173-updates (173.14.35) or Nouveau open-source driver, the free() error is the same, but the backtrace is different:
+
+ *** glibc detected *** MP4Box: free(): invalid next size (fast): 0x00000000022d2420 ***
+ ======= Backtrace: =========
+ /lib/x86_64-linux-gnu/libc.so.6(+0x7e626)[0x7f5e9820a626]
+ /usr/lib/x86_64-linux-gnu/libgpac.so.1(minf_del+0x3f)[0x7f5e9867c69f]
+ /usr/lib/x86_64-linux-gnu/libgpac.so.1(mdia_del+0x25)[0x7f5e9867c395]
+ /usr/lib/x86_64-linux-gnu/libgpac.so.1(trak_del+0x33)[0x7f5e98680c03]
+ /usr/lib/x86_64-linux-gnu/libgpac.so.1(gf_isom_box_array_del+0x37)[0x7f5e98692c87]
+ /usr/lib/x86_64-linux-gnu/libgpac.so.1(moov_del+0x58)[0x7f5e9867c9c8]
+ /usr/lib/x86_64-linux-gnu/libgpac.so.1(gf_isom_box_array_del+0x37)[0x7f5e98692c87]
+ /usr/lib/x86_64-linux-gnu/libgpac.so.1(gf_isom_delete_movie+0x3a)[0x7f5e9869ad5a]
+ /usr/lib/x86_64-linux-gnu/libgpac.so.1(gf_isom_close+0x39)[0x7f5e9869c779]
+ MP4Box[0x40d30c]
+ /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7f5e981ad76d]
+ MP4Box[0x4085c1]
+
+The problem appears to be intermittent: two successive runs can produce different results. It is also noticeable immediately after a reboot. At this time I'm not sure whether it's system memory or an issue with MP4Box/gpac. There were similar crashing issues in the Windows version, which is why multiple versions of MP4Box were bundled and used for fallback.
+
+If you do see this issue in your own testing, please report it and include a link to the file that causes the problem if possible. You may also be able to get the file to convert by using different command line options, such as *--resume-previous --preserve-temp-files* or including or excluding *-vv*.
+
+Relevant system information:
+
+ $ uname -a
+ Linux ubuntu 3.2.0-29-generic #46-Ubuntu SMP Fri Jul 27 17:03:23 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
+
+My MP4Box version is the default from the 'gpac' Ubuntu 12.04 package, which is 0.4.5+svn3462~dfsg0-1:
+
+ $ MP4Box -version
+ MP4Box - GPAC version 0.4.6-DEV-rev
+ GPAC Copyright: (c) Jean Le Feuvre 2000-2005
+ (c) ENST 2005-200X
+ GPAC Configuration: --build=x86_64-linux-gnu --prefix=/usr --includedir=${prefix}/include --mandir=${prefix}/share/man --infodir=${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --libdir=${prefix}/lib/x86_64-linux-gnu --libexecdir=${prefix}/lib/x86_64-linux-gnu --disable-maintainer-mode --disable-dependency-tracking --prefix=/usr --mandir=${prefix}/share/man --libdir=lib/x86_64-linux-gnu --extra-cflags='-Wall -fPIC -DPIC -I/usr/include/mozjs -DXP_UNIX' --enable-joystick --enable-debug --disable-ssl
+ Features: GPAC_HAS_JPEG GPAC_HAS_PNG
+
+There is a 0.5.0 version available at <https://sourceforge.net/projects/gpac/> that may also be worth building and trying. The version included with Ubuntu 10.04 (0.4.5-0.3ubuntu6) also does not appear to trigger this bug.
+
+# TODO
+
+* Add a ConfigParser instance, allowing default options to be read and stored in persistent files: <http://docs.python.org/library/configparser.html>
+* Add support for custom tool locations (eg: use newer MP4Box build in a custom directory, rather than the version defined in PATH)
+* Add support for picking a specific track from a multiple-track MKV file, rather than forcing defaults
View
2 decoder.py
@@ -47,7 +47,7 @@ def decode_mplayer(self):
self.log.debug("Deleting temporary mplayer output file %s/audiodump.wav" % os.getcwd())
os.unlink(os.getcwd() + "/audiodump.wav")
- cmd = ["mplayer", self.file_path, "-benchmark", "-vc", "null", "-vo", "null", "-channels", "2", "-ao", "pcm:fast"]
+ cmd = ["mplayer", self.file_path, "-benchmark", "-vc", "null", "-vo", "null", "-channels", "2", "-noautosub", "-ao", "pcm:fast"]
process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
while True:
View
33 file_utils.py
@@ -1,10 +1,32 @@
import os
+import sys
class FileUtils:
- log = None
+ log = args = None
+ FOUR_GIGS = (4*1024*1024*1024)
- def __init__(self, log):
+ def __init__(self, log, args):
self.log = log
+ self.args = args
+
+ # Check if all dependent applications are installed on the system and present in PATH
+ # TODO: Allow custom path to be specified for each of these if it is in a config file
+ def check_dependencies(self):
+ dependencies = ('mkvinfo', 'mediainfo', 'mkvextract', 'mplayer', 'faac', 'MP4Box')
+ ospath = os.defpath.split(os.pathsep)
+ for app in dependencies:
+ app_present = False
+ for path in ospath:
+ if os.path.isfile(os.path.join(path, app)):
+ self.log.debug("Found dependent application %s in %s" % (app, path))
+ app_present = True
+ break
+
+ if not app_present:
+ self.log.critical("Dependent application '%s' was not found in PATH. Please make sure it is installed." % app)
+ sys.exit(1)
+
+ return True
def check_dest_dir(self, destination):
if not os.path.isdir(destination):
@@ -15,6 +37,13 @@ def check_source_file(self, source_file):
if not os.path.isfile(source_file):
log.critical("Source file %s does not exist" % source_file)
sys.exit(1)
+
+ filesize = os.path.getsize(source_file)
+ if (filesize >= self.FOUR_GIGS):
+ log.warning("File size of %s is %i, which is over 4GiB. This file may not play on certain devices and cannot be copied to a FAT32-formatted storage medium." % (source_file, filesize))
+ if self.args.error_filesize:
+ log.critical("Cancelling processing as file size limit of 4GiB is exceeded")
+ sys.exit(1)
def hex_edit_video_file(self, path):
with open(path, 'r+b') as f:
View
2 reference_frame.py
@@ -36,7 +36,7 @@ def recurse_width(frame_list, index, height, reference_frames):
current_test = frame_list[index]
if height < current_test[0]:
- return recurse_width(frame_list, index + 1, height, reference_frames)
+ return ReferenceFrameValidator.recurse_width(frame_list, index + 1, height, reference_frames)
elif height == current_test[0]:
return reference_frames > current_test[1]
else: # height > current_test[0]:
View
113 xenonmkv.py
@@ -4,9 +4,17 @@
# Jake Billo, jake@jakebillo.com
# https://github.com/jbillo/xenonmkv
-import argparse
+
import sys
import os
+
+# Check for Python version before running argparse import
+if sys.version_info[0] == 2 and sys.version_info[1] < 7:
+ print "You need to be running at least Python 2.7 to use this application."
+ print "Try running /usr/bin/python2.7 %s" % sys.argv[0]
+ sys.exit(1)
+
+import argparse
import subprocess
import logging
import fractions
@@ -239,9 +247,14 @@ def parse_mkvinfo(self, result):
track.language = mediainfo_track[1]
track.channels = int(mediainfo_track[2])
- # Preemptively indicate if the audio track needs a recode
- # If we have an AAC file with two channels, nothing needs to happen
- track.needs_recode = (track.codec_id != "A_AAC" or track.channels != 2)
+ # Indicate if the audio track needs a recode. By default, it does.
+ # Check that the audio type is AAC, and if the number of channels in the file
+ # is less than or equal to what was specified on the command line, no recode is necessary
+ if track.codec_id == "A_AAC":
+ # Check on the number of channels in the file versus the argument passed.
+ if track.channels <= args.channels:
+ log.debug("Audio track %i will not need to be re-encoded (%s channels specified, %i channels in file)" % (track.number, args.channels, track.channels))
+ track.needs_recode = False
log.debug("Audio track %i has codec %s and language %s" % (track.number, track.codec_id, track.language))
log.debug("Audio track %i has %i channel(s)" % (track.number, track.channels))
@@ -331,18 +344,18 @@ def extract_mkv(self):
if args.resume_previous and os.path.isfile(temp_video_file) and os.path.isfile(temp_audio_file):
log.debug("Temporary video and audio files already exist; cancelling extract")
- temp_video_file = os.getcwd() + "/" + temp_video_file
- temp_audio_file = os.getcwd() + "/" + temp_audio_file
+ temp_video_file = os.path.join(os.getcwd(), temp_video_file)
+ temp_audio_file = os.path.join(os.getcwd(), temp_audio_file)
os.chdir(prev_dir)
return (temp_video_file, temp_audio_file)
# Remove any existing files with the same names
if os.path.isfile(temp_video_file):
- log.debug("Deleting temporary video file %s" % os.getcwd() + "/" + temp_video_file)
+ log.debug("Deleting temporary video file %s" % os.path.join(os.getcwd(), temp_video_file))
os.unlink(temp_video_file)
if os.path.isfile(temp_audio_file):
- log.debug("Deleting temporary audio file %s" % os.getcwd() + "/" + temp_audio_file)
+ log.debug("Deleting temporary audio file %s" % os.path.join(os.getcwd(), temp_audio_file))
os.unlink(temp_audio_file)
video_output = str(self.video_track_id) + ":" + temp_video_file
@@ -363,8 +376,8 @@ def extract_mkv(self):
log.critical("An error occurred while extracting tracks from %s - please make sure this file exists and is readable" % self.get_path())
sys.exit(1)
- temp_video_file = os.getcwd() + "/" + temp_video_file
- temp_audio_file = os.getcwd() + "/" + temp_audio_file
+ temp_video_file = os.path.join(os.getcwd(), temp_video_file)
+ temp_audio_file = os.path.join(os.getcwd(), temp_audio_file)
os.chdir(prev_dir)
log.debug("mkvextract finished; attempting to parse output")
@@ -395,6 +408,10 @@ def package(self):
sys.stdout.write(out)
sys.stdout.flush()
+ if process.returncode != 0:
+ log.critical("An error occurred while creating an MP4 file with MP4Box")
+ sys.exit(1)
+
log.debug("MP4Box process complete")
# When complete, change back to original directory
@@ -411,11 +428,14 @@ def package(self):
parser.add_argument('-nrp', '--no-round-par', help='When processing video, do not round pixel aspect ratio from 0.98 to 1.01 to 1:1.', action='store_true')
parser.add_argument('-irf', '--ignore-reference-frames', help='If the source video has too many reference frames to play on low-powered devices (Xbox, PlayBook), continue converting anyway', action='store_true')
parser.add_argument('-sd', '--scratch-dir', help='Specify a scratch directory where temporary files should be stored (default: /var/tmp/)', default='/var/tmp/')
+parser.add_argument('-c', '--channels', help='Specify the maximum number of channels that are acceptable in the output file. Certain devices (Xbox) will not play audio with more than two channels. If the audio needs to be re-encoded at all, it will be downmixed to two channels only. Possible values for this option are 2 (stereo); 4 (surround); 5.1 or 6 (full 5.1); 7.1 or 8 (full 7.1 audio). For more details, view the README file.', default="6")
parser.add_argument('-fq', '--faac-quality', help='Quality setting for FAAC when encoding WAV files to AAC. Defaults to 150 (see http://wiki.hydrogenaudio.org/index.php?title=FAAC)', default=150)
parser.add_argument('-rp', '--resume-previous', help='Resume a previous run (do not recreate files if they already exist). Useful for debugging quickly if a conversion has already partially succeeded.', action='store_true')
parser.add_argument('-n', '--name', help='Specify a name for the final MP4 container. Defaults to the original file name.', default="")
parser.add_argument('-st', '--select-tracks', help="If there are multiple tracks in the MKV file, prompt to select which ones will be used. By default, the last video and tracks flagged as 'default' in the MKV file will be used.", action='store_true')
parser.add_argument('-preserve', '--preserve-temp-files', help="Preserve temporary files on the filesystem rather than deleting them at the end of each run.", action='store_true')
+parser.add_argument("-p", '--profile', help="Select a standardized device profile for encoding. Current profile options are: xbox360, playbook", default="")
+parser.add_argument("-eS", "--error-filesize", help="Stop processing this file if it is over 4GiB. Files of this size will not be processed correctly by some devices such as the Xbox 360, and they will not save correctly to FAT32-formatted storage. By default, you will only see a warning message, and processing will continue.", action="store_true")
if len(sys.argv) < 2:
parser.print_help()
@@ -428,36 +448,61 @@ def package(self):
log.addHandler(console_handler)
args = parser.parse_args()
-source_file = args.source_file
+# Depending on the arguments, set the logging level appropriately.
if args.quiet:
log.setLevel(logging.ERROR)
elif args.very_verbose:
log.setLevel(logging.DEBUG)
log.debug("Using debug/very verbose mode output")
elif args.verbose:
log.setLevel(logging.INFO)
+
+# Check for 5.1/7.1 audio with the channels setting
+if args.channels == "5.1":
+ args.channels = "6"
+elif args.channels == "7.1":
+ args.channels = "8"
+if args.channels not in ('2', '4', '6', '8'):
+ log.warning("An invalid number of channels was specified. Falling back to 2-channel stereo audio.")
+ args.channels = "2"
+
+if args.profile:
+ if args.profile == "xbox360":
+ args.channels = "2"
+ args.error_filesize = True
+ elif args.profile == "playbook":
+ args.channels = "6"
+ args.error_filesize = False
+ else:
+ log.warning("Unrecognized device profile %s" % args.profile)
+ args.profile = ""
log.debug("Starting XenonMKV")
-f_utils = FileUtils(log)
# Check if we have a full file path or are just specifying a file
-if "/" not in source_file:
- log.debug("Ensuring that we have a complete path to %s" % source_file)
- source_file = os.getcwd() + "/" + source_file
- log.debug("%s will be used to reference the original MKV file" % source_file)
-
+if os.sep not in args.source_file:
+ log.debug("Ensuring that we have a complete path to %s" % args.source_file)
+ args.source_file = os.path.join(os.getcwd(), args.source_file)
+ log.debug("%s will be used to reference the original MKV file" % args.source_file)
+
# Always ensure destination path ends with a slash
-if not args.destination.endswith('/'):
- args.destination += '/'
+if not args.destination.endswith(os.sep):
+ args.destination += os.sep
+
+if not args.scratch_dir.endswith(os.sep):
+ args.scratch_dir += os.sep
+
+# Initialize file utilities
+f_utils = FileUtils(log, args)
-if not args.scratch_dir.endswith('/'):
- args.scratch_dir += '/'
+# Check if all dependent applications are installed and available in PATH
+f_utils.check_dependencies()
-# Check if source file exists
-f_utils.check_source_file(source_file)
+# Check if source file exists and is an appropriate size
+f_utils.check_source_file(args.source_file)
-source_basename = os.path.basename(source_file)
+source_basename = os.path.basename(args.source_file)
source_noext = source_basename[0:source_basename.rindex(".")]
if not args.name:
@@ -467,14 +512,14 @@ def package(self):
# Check if destination directory exists
f_utils.check_dest_dir(args.destination)
-log.info("Loading source file %s " % source_file)
+log.info("Loading source file %s " % args.source_file)
-to_convert = MKVFile(source_file)
+to_convert = MKVFile(args.source_file)
to_convert.get_mkvinfo()
# Check for multiple tracks
if to_convert.has_multiple_av_tracks():
- log.debug("Source file %s has multiple audio and/or video tracks" % source_file)
+ log.debug("Source file %s has multiple audio and/or video tracks" % args.source_file)
if args.select_tracks:
# TODO: Add selector for tracks
# Handle this scenario: prompt the user to select specific tracks,
@@ -486,7 +531,7 @@ def package(self):
to_convert.set_default_av_tracks()
else:
# Pick default (or only) audio/video tracks
- log.debug("Source file %s has 1 audio and 1 video track; using these" % source_file)
+ log.debug("Source file %s has 1 audio and 1 video track; using these" % args.source_file)
to_convert.set_default_av_tracks()
# Next phase: Extract MKV files to scratch directory
@@ -502,14 +547,14 @@ def package(self):
# Detect which audio codec is in place and dump audio to WAV accordingly
if to_convert.get_audio_track().needs_recode:
- log.debug("Audio track %s needs to be re-encoded to a 2-channel AAC file" % audio_file)
+ log.debug("Audio track %s needs to be re-encoded" % audio_file)
audio_dec = AudioDecoder(audio_file, log, args)
audio_dec.decode()
# Once audio has been decoded to a WAV, use the FAAC application to encode it to .aac
- faac_enc = FAACEncoder(args.scratch_dir + "audiodump.wav", log, args)
+ faac_enc = FAACEncoder(os.path.join(args.scratch_dir, "audiodump.wav"), log, args)
faac_enc.encode()
- encoded_audio = args.scratch_dir + "audiodump.aac"
+ encoded_audio = os.path.join(args.scratch_dir, "audiodump.aac")
else:
# Bypass this whole encoding shenanigans and just reference the already-valid audio file
encoded_audio = audio_file
@@ -520,10 +565,10 @@ def package(self):
mp4box.package()
# Move the file to the destination directory with the original name
-dest_path = args.destination + source_noext + ".mp4"
-os.rename(args.scratch_dir + "output.mp4", dest_path)
+dest_path = os.path.join(args.destination, source_noext + ".mp4")
+os.rename(os.path.join(args.scratch_dir, "output.mp4"), dest_path)
-log.info("Processing of %s complete; file saved as %s" % (source_file, dest_path))
+log.info("Processing of %s complete; file saved as %s" % (args.source_file, dest_path))
# Delete temporary files if possible
if not args.preserve_temp_files:

0 comments on commit 23b20d2

Please sign in to comment.