Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Python Prolog Perl
branch: mysql

This branch is 1 commit ahead, 67 commits behind master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
sql
.bzrignore
README.txt
mbslave-import.py
mbslave-psql.py
mbslave-sync.py
mbslave.conf.default
update-sql.sh

README.txt

Installation
============

1. Setup a database and create mbslave.conf by copying and editing
   mbslave.conf.default. If you are starting completely from scratch,
   you can use the following commands to setup a clean database:

   $ sudo su - postgres
   $ createuser musicbrainz
   $ createdb -O musicbrainz musicbrainz
   $ createlang plpgsql musicbrainz

2. Prepare empty schema for the MusicBrainz database (skip this if you
   want to use the default 'public' schema), install the cube extension into
   this new schema and create the table structure:

   $ echo 'CREATE SCHEMA musicbrainz;' | ./mbslave-psql.py
   $ sed 's/public/musicbrainz/' `pg_config --sharedir`/contrib/cube.sql | psql -U postgres musicbrainz
   $ ./mbslave-psql.py <sql/CreateTables.sql

3. Download the MusicBrainz database dump files from
   http://ftp.musicbrainz.org/pub/musicbrainz/data/fullexport/

4. Import the data dumps, for example:

   $ ./mbslave-import.py mbdump.tar.bz2 mbdump-derived.tar.bz2

5. Setup primary keys, indexes, views and functions:

   $ ./mbslave-psql.py <sql/CreatePrimaryKeys.sql
   $ ./mbslave-psql.py <sql/CreateFunctions.sql
   $ grep -vE '(collate|page_index)' sql/CreateIndexes.sql | ./mbslave-psql.py
   $ ./mbslave-psql.py <sql/CreateViews.sql

6. Vacuum the newly created database (optional)

   $ echo 'VACUUM ANALYZE;' | ./mbslave-psql.py

7. Run the initial replication:

   $ ./mbslave-sync.py

Replication
===========

In order to regularly update your database with the latest changes, setup
a cron job like this that runs every hour:

15 * * * * $HOME/mbslave/mbslave-sync.py >>/var/log/mbslave.log

Upgrading
=========

Release 2011-07-13
~~~~~~~~~~~~~~~~~~

 $ ./mbslave-psql.py <sql/updates/20110624-cdtoc-indexes.sql
 $ ./mbslave-psql.py <sql/updates/20110710-tracklist-index-slave-before.sql
 $ echo "TRUNCATE url_gid_redirect" | ./mbslave-psql.py
 $ echo "TRUNCATE work_alias" | ./mbslave-psql.py
 $ curl -O "ftp://data.musicbrainz.org/pub/musicbrainz/data/20110711-update.tar.bz2"
 $ curl -O "ftp://data.musicbrainz.org/pub/musicbrainz/data/20110711-update-derived.tar.bz2"
 $ ./mbslave-import.py 20110711-update.tar.bz2 20110711-update-derived.tar.bz2
 $ ./mbslave-psql.py <sql/updates/20110710-tracklist-index-slave-after.sql
 $ echo "UPDATE replication_control SET current_schema_sequence = 13, current_replication_sequence = 51420;" | ./mbslave-psql.py

Optionally, if you want to integrate tables from the old rawdata database,
in case they start being replicated in the future, you can also run these
commands:

 $ ./mbslave-psql.py <sql/vertical/rawdata/CreateTables.sql
 $ ./mbslave-psql.py <sql/vertical/rawdata/CreateIndexes.sql
 $ ./mbslave-psql.py <sql/vertical/rawdata/CreatePrimaryKeys.sql
 $ ./mbslave-psql.py <sql/vertical/rawdata/CreateFunctions.sql
 $ echo "ALTER TABLE edit_artist ADD status smallint NOT NULL;" | ./mbslave-psql.py
 $ echo "CREATE INDEX edit_artist_idx_status ON edit_artist (status);" | ./mbslave-psql.py
 $ echo "ALTER TABLE edit_label ADD status smallint NOT NULL;" | ./mbslave-psql.py
 $ echo "CREATE INDEX edit_label_idx_status ON edit_label (status);" | ./mbslave-psql.py

MySQL Installation
==================

CREATE USER 'musicbrainz'@'localhost';
CREATE DATABASE musicbrainz;
GRANT ALL PRIVILEGES ON musicbrainz.* TO 'musicbrainz'@'localhost';

sed 's/SERIAL/INTEGER/' sql/CreateTables.sql \
	| sed 's/TIMESTAMP WITH TIME ZONE/TIMESTAMP/' \
	| sed 's/CUBE/TEXT/' \
	| sed 's/UUID/CHAR(36)/' \
	| sed 's/VARCHAR /TEXT /' \
	| sed 's/INTEGER\[\]/TEXT/' \
	| sed 's/ release / `release` /' \
	| sed 's/ DEFAULT NOW()//' \
	| sed 's/INTERVAL/VARCHAR(100)/' \
	| sed 's/);/) ENGINE=InnoDB;/' \
	| grep -v 'ON_ERROR_STOP' \
	| mysql -u musicbrainz musicbrainz

Something went wrong with that request. Please try again.