Skip to content

Library

Joshua Haas edited this page Feb 21, 2017 · 13 revisions

This plug-in maintains a media "library" of audio and video files and directories to speed up searching. The first time you start Sibyl, it will browse through everything in library.audio_dirs and library.video_dirs to build the library. Sibyl will not be able to find any files that are created after the library is built. To find new files, simply use the library rebuild command.

Dependencies

  • smbc - (OPTIONAL) python samba bindings - pip install pysmbc

Config

  • library.file - file in which to store the library (default data/library.pickle)
  • library.audio_dirs - directories to search for audio files (default [] i.e. none)
  • library.video_dirs - directories to search for video files (default [] i.e. none)
  • library.max_matches - max number of search results to return (default 10; 0 is unlimited)
  • library.remote - used when running sibyl on a different box than Kodi

Commands

  • library: control media library - library (info|load|rebuild|save)
  • search: search all paths for matches - search [include -exclude]

library

sibyl library (info|load|rebuild|save|reload)

Sibyl maintains a list of files and folders to speed up searching referred to as its "library". You can use this function to interact with it. The library is saved using python's pickle module.

  • info - Respond with the last time the library was updated
  • load - Load the library from the pickle file (default: sibyl.pickle)
  • rebuild - Traverse all search directories, build the library, then save it
  • save - Save the library to a pickle file (default: sibyl.pickle)
  • reload - Reload the library.video_dirs and library.audio_dirs config options

search

sibyl search search1 [search2 search3 ...]

Same as audio but searches both audio_dirs and video_dirs for matching files and folders. This command does not interact with XBMC.

Specifying Library Search Paths

The parameters library.audio_dirs and library.video_dirs are lists of directories. Each entry can either be a local path on the machine running sibyl, or a samba share. In the below example the first line is a local folder, the second is a samba share that does not require a password, and the last is a samba share that needs a password. Each entry must be separated by a semi-colon.

library.video_dirs = /media/flashdrive/videos;
                     MEDIA,videos;
                     LAPTOP,videos,user,pass

It shouldn't be a big deal if there are audio files in the folders in library.video_dirs or if there are video files in the folders in library.audio_dirs. They will likely play without issue. The separation between types exists due to both the way XBMC interacts with file types (it has different players and playlists) and for commands that want to distinguish between the two.

If you have network shares/mounts that aren't available during Sibyl's startup, they will be ignored during config parsing. If you fix them while Sibyl is running, you can reload library.video_dirs and library.audio_dirs with sibyl library reload. If there are new directories, you'll still have to do sibyl library rebuild to index them.

Remote Kodi Instance

If you are running Sibyl on a different computer than Kodi, you have two choices for using this plugin. First, you can make sure all library paths are the same on both machines. For example, make sure the Sibyl box has your Music folder mounted at /mnt/music and your Kodi box also has your Music folder mounted at /mnt/music.

NOTE: None of this is necessary for samba shares, unless you mount them via fstab, autofs, etc.

The second option is to keep your existing mount points, and use the library.remote config option to translate between the two hosts. The format is simply:

library.remote = localpath1,remotepath1; localpath2,remotepath2

For example, maybe you have a flash drive connected to the machine running Kodi mounted at /media/pi/flash/. You have Sibyl running on another computer, and to give it access to that flash drive so it can build its internal library, you use sshfs to mount the flash drive as /mnt/piflash/ on Sibyl's machine. Now you have the following (only the last 2 lines would go in your config file):

localpath (relative to sibyl): /mnt/piflash/
remotepath (relative to Kodi): /media/pi/flash/

library.video_dirs = /mnt/piflash
library.remote = /mnt/piflash, /media/pi/flash

So when Sibyl builds the library it will see for example /mnt/piflash/song.mp3. But when sending a request to kodi, it will replace /mnt/piflash with /media/pi/flash resulting in /media/pi/flash/song.mp3 which is how Kodi accesses and plays the file.

Now, let's say you have an external hard drive connected to the machine running Kodi, and it's mounted at /media/pi/external/. You could give Sibyl access using samba (click here for more details on setting up samba). For this example let's call your Kodi host Kodi and the samba share External. We have the following (again only the last two lines are config lines):

localpath (relative to sibyl): smb://Kodi/External
remotepath (relative to Kodi): /media/pi/external

library.video_dirs = Kodi,External
library.remote = smb://Kodi/External, /media/pi/external

Sibyl stores samba paths internally as smb://host/share so when sending a request to kodi, sibyl will change smb://Kodi/External/movie.mkv to /media/pi/external/movie.mkv.

Unicode Considerations

If you have file names that use characters outside of the ASCII set, you need to make certain your locale is set correctly. Barring that, you can force it manually by starting Sibyl for example with LANG=en_US.UTF-8 python run.py. You can check how python is interpreting file names with:

python -c "import sys; print sys.getfilesystemencoding()"

If it returns something other than UTF-8 then you're likely to have a problem. You can list your installed locales with:

locale -a

Pick one that ends in utf8 and makes sense based on your location/language. Let's say you're going to use de_DE.utf8. You can test that everything is working correctly by making sure the following returns UTF-8:

LANG=de_DE.utf8 python -c "import sys; print sys.getfilesystemencoding()"

Both the init script init/sibyl.init and the systemd unit init/sibyl.service in this repository already set the locale manually to en_US.utf8. If you do not have that locale, you'll need to make these changes depending on which init method you're using. We'll use de_DE.utf8 as the example again.

# For the systemd service find the line
Environment=LANG=en_US.UTF-8
# and replace it with
Environment=LANG=de_DE.utf8

# For the init script find the line
export LANG=und.UTF-8
# and replace it with
export LANG=de_DE.utf8

Advanced Info

Sibyl stores the library internally in four variables: lib_audio_dir, lib_audio_file, lib_video_dir, and lib_video_file. These are just lists of file paths. It also stores how long the last library rebuild took in lib_last_elapsed and the timestamp of the last rebuild in lib_last_rebuilt. Additionally, the library is saved in the file specified by lib_file (default: data/sibyl_lib.pickle) using python's pickle module. The pickle module very conveniently saves python variables and objects to a file from which they can later be retrieved and be instantly usable. All previously mentioned variables are stored in a dict using str keys of the same name before being pickled.

If you want to manually inspect the library, you can do so easily via the python interpreter. For example:

import pickle

with open('sibyl_lib.pickle','r') as f:
  lib = pickle.load(f)

lib.keys()
lib['lib_last_rebuilt']
Clone this wiki locally