Setup

Installation

Git

If you want the latest code or even feel like contributing, the code is available on Github.

You can easily clone the code with git:

git clone git://github.com/metabrainz/sir.git

Now you can install it system-wide:

python2 setup.py install

or start hacking on the code. To do that, you’ll need to run at least:

python2 setup version

once to generate the file sir/version.py which the code needs. This file does not have to be added into the git repository because it only contains the hash of the current git commit, which changes after each commit operation.

Setup

The easiest way to run sir at the moment is to use a virtual environment. Once you have virtualenv for Python 2.7 installed, use the following to create the environment:

virtualenv venv
source venv/bin/activate
pip install -r requirements.txt
cp config.ini.example config.ini

Note: Environment variables can be used in config.ini with the syntaxes $NAME and ${NAME}. Undefined variables will not be replaced at all. Escaping is not supported.

You can now use sir via:

python -m sir

AMQP Setup

RabbitMQ Server

To set up the exchanges and queues on your RabbitMQ server,

  • install RabbitMQ (if you have not already done so)
  • start RabbitMQ
  • configure your AMQP access data in config.ini
  • run python -m sir amqp_setup to configure the necessary exchanges and queues on your AMQP server.

The default values for the RabbitMQ configuration options can be found in the RabbitMQ documentation.

Database

Sir requires that you both install an extension into your MusicBrainz database and add triggers to it.

It also requires to have built the materialized (or denormalized) tables for the MusicBrainz database.

AMQP Extension

  • Install pg_amqp.
  • Check values for the following keys in the file config.ini:
Keys Description
[database] user Name of the PostgreSQL user the MusicBrainz Server uses
[rabbitmq] host The hostname that’s running your RabbitMQ server
[rabbitmq] user The username with which to connect to your RabbitMQ server
[rabbitmq] password The password with which to connect to your RabbitMQ server
[rabbitmq] vhost The vhost on your RabbitMQ server

The default values for the RabbitMQ configuration options can be found in the RabbitMQ documentation.

  • Run python -m sir extension once to generate the file sql/CreateExtension.sql.
  • Connect to your database as a superuser with psql to execute from this file.

Triggers

In addition to the steps above, it is necessary to install functions and triggers into the database to send messages via AMQP after a change has been applied to the database. Those can be found in the sql directory and will send messages for all entity types by default.

If you just want search indices to be updated for a limited set of entity types, for example artists and works, you can regenerated those by running

python -m sir triggers --entity-type artist --entity-type work

Once you are satisfied with the (default or generated) SQL triggers, those can be installed with

MB_SERVER_PATH=<mb_path> make installsql

where <mb_path> is the path to your clone of the MusicBrainz server.

Solr

Of course you’ll need a Solr server somewhere to send the data to. The mbsssss repository contains instructions on how to add the MusicBrainz schemas to a Solr server.

MusicBrainz Database Schema

Of course you’ll need a MusicBrainz database somewhere to read the data from. The active database schema sequence must be 27 (or any future schema version if still compatible). Follow announcements from the MetaBrainz blog.

Only Sir 3.y.z is able to read from database of schema sequence 27 (or any future schema if still compatible, but it reads and sends the data made available from schema sequence 27 only).

Web Service Compatibility

If you have applications that are already able to parse search results from search.musicbrainz.org in the mmd-schema XML or the derived JSON format, you can enable the wscompat setting in the configuration file. This will store an mmd-compatible XML document in a field called _store for each Solr document. Installing mb-solrquerywriter on your Solr server will then allow you to retrieve responses as mmd-compatible XML or the derived JSON.