Just to get documentation started, a quick copy/paste from the old source - BGD.

 

 

How To Setup MogileFS

Version: 0.06, October 1, 2009

Author: Brett G. Durrett (first name at last name dot net)

Source: http://durrett.net/mogilefs_setup.html

 

 

Overview

 

This document explains how to set up and configure a basic MogileFS installation. It is intended for the novice systems administrator and should enable anybody with the skills to install a basic Linux distro to get MogileFS up and running on it.

 

I am going to assume you have three roles for this setup. A machine should be able to handle more than one role. I will use hosts mogiledb.yourdomain.com, mogiletracker.yourdomain.com and mogilestorage.yourdomain.com – rename these to meet your needs.

 

 

Getting MogileFS

 

If you look around you can find MogileFS directly on the Danga website, on CPAN and in the current SVN repository. If you pull from SVN you will have a shiny new version that works best with this documentation.

 

Using svn (http://subversion.tigris.org/), checkout the latest and greatest version (the example below pulls from the trunk).

 

$ mkdir mogilefs-src
$ cd mogilefs-src
$ svn checkout http://code.sixapart.com/svn/mogilefs/trunk

 

 

Installation

 

 

Creating a Database

 

Setting up a MySQL server is beyond the scope of this document – there are packages available for most distributions, find one that suits your needs. Once you have a MySQL server up and running on host mogiledb.yourdomain.com, create a table and user for MogileFS. Some libraries don't play well with new MySQL passwords – if you use these, set the password using the “OLD_PASSWORD” function. Make sure you change the password to something better than the example.

 

# mysql
mysql> CREATE DATABASE mogilefs;
mysql> GRANT ALL ON mogilefs.* TO 'mogile'@'%';
mysql> SET PASSWORD FOR 'mogile'@'%' = OLD_PASSWORD( 'sekrit' );
mysql> FLUSH PRIVILEGES;
mysql> quit

 

You will also need to create the schema – that is covered later in this document. 

 

 

Setting up the Trackers and Storage Servers

 

You need to obtain and install the MogileFS-specific perl modules and their dependencies before MogileFS will install successfully. Hopefully you pulled the MogileFS source from the SVN repository – if so, the server components will be located in the trunk/server directory. You probably need to install dependent modules first (see below) but once all dependencies have been met, you can install Mogile with the following commands (starting in the top of the SVN directory you pulled):

 

# cd trunk/server/
# perl Makefile.PL
# make
# make test
# make install

 

If you get any errors during this process, they will probably be errors telling you that a dependent module is missing. If during the 'make test' step you get the error, "t/00-startup....DBI connect('mysql','root',...) failed: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at t/lib/mogtestlib.pl line 16" it can probably be ignored – it should not be necessary for MySQL to be running on any host other than mogiledb.yourdomain.com.

You probably want to install some helpful utilities on each tracker or storage server as well (these will be needed for later configuration). These are located in the trunk/utils directory and can be installed with the following commands (starting in the top of the SVN directory you pulled):

 

# cd trunk/utils/
# perl Makefile.PL
# make
# make test
# make install

 

You also want the API – the utilities will require this. These are located in the trunk/api/perl directory and can be installed with the following commands (starting in the top of the SVN directory you pulled):

 

# cd trunk/api/perl
# perl Makefile.PL
# make
# make test
# make install

 

 

Installing Perl Module Dependencies

 

As for Perl modules, you need (at least) Perlbal and Danga-Socket and any dependencies. If you are not sure how to locate the dependent modules, consider using CPAN (http://cpan.org/) as it can install a module and all of the dependencies. If you already have a reasonable Perl installation you probably have most of the non-MogileFS modules already.

The basic process for installing most perl modules is:

 

Extract the module:

 

# tar -xvzf Linux-AIO-1.9.tar.gz

 

Create the makefile:

 

# cd Linux-AIO-1.9
# perl Makefile.PL

 

Build the code:

 

# make

 

Test the code (some modules may not have this step but it is okay to run anyway):

 

# make test

 

Install it:

 

# make install

 

If you see any errors stating that a module dependency is missing, obtain and install the missing module first and try again.

 

 

Setup

 

 

Database Configuration

 

The database is empty and will need a schema applied. The 'trunk/server' directory has a utility named 'mogdbsetup' to make this process simple. By default it assumes the database is located on localhost, so if you are running it from a different host you will need to provide the host name on the command line.

 

# ./mogdbsetup --dbhost=mogiledb.yourdomain.com --dbname=mogilefs --dbuser=mogile --dbpassword=sekrit

 

Again, make sure you replace the host and password so that they match your database configuration from above.

 

The mogdbsetup utility does not specify a table type by default ,so your tables will match the defaults for your database. In many cases this will mean that you end up with MyISAM tables. If you prefer InnoDB tables you will either need to make sure your database defaults to InnoDB, or you can manually convert the tables (both of these are outside of the scope of this document but there are plenty of examples out there).

 

 

 

Tracker Configuration

 

On each tracker server (mogiletracker.yourdomain.com), create a configuration file at /etc/mogilefs/mogilefsd.conf with the following:

 

db_dsn DBI:mysql:mogilefs:mogiledb.yourdomain.com
db_user mogile
db_pass sekrit
conf_port 6001
listener_jobs 5

 

db_dsn points to your database instance. If you are running the database on the same machine as the storage server, you can omit ":mogiledb.yourdomain.com: and it will use the local machine. db_user and db_pass should match the user and password you configured when setting up your database.

 

The program 'mogilefsd' will not run as root, so you will need to run this as a non-root user. To create a user for this, enter the following command and follow the prompts to create the "mogile" user:

 

# adduser mogile

 

In order to use the tools to set up the storage servers, you will need to have the trackers running. Refer to "Starting Trackers", below. 

 

 

Storage Server Configuration

 

On each storage server, create the storage directory (make sure it has access permissions for the user you will use to run mogstored):

 

# mkdir /var/mogdata

 

Configure it:

On each storage server, create a configuration file at /etc/mogilefs/mogstored.conf with the following:

 

httplisten=0.0.0.0:7500
mgmtlisten=0.0.0.0:7501
docroot=/var/mogdata

 

Use 'mogadm' to add each storage server to the database. This requires that the trackers are already running so if you have not already started them, refer to "Starting Trackers", below. You need to supply the Perl lib path which has the 'MogileFS.pm' perl module installed – this was installed if you installed the API in the "Setting up the Trackers and Storage Servers" section above. The following example would add the host mogilestorage.yourdomain.com as a storage server, assuming that mogilestorage.yourdomain.com had an IP address of 192.168.42.3 (listening on port 7500) and your tracker had an IP address of 192.168.42.1 (listening on port 6001):

 

# mogadm --lib=/usr/local/share/perl/5.8.4 --trackers=192.168.42.1:6001 host add mogilestorage --ip=192.168.42.3 --port=7500 --status=alive

 

You can confirm that your host(s) were added with the following command;

 

# mogadm --lib=/usr/local/share/perl/5.8.4 --trackers=192.168.42.1:6001 host list

 

You also need to add devices for each storage host. If you don't provide a device id mogadm is supposed to do it for you... in practice I find it dies with an error, so you will need to manually add a unique device id after the host: 

 

# mogadm --lib=/usr/local/share/perl/5.8.4 --trackers=192.168.42.1:6001 device add mogilestorage 1

 

Finally, add the correctly-name device (folder) to each storage host. I have been unable to get the tools to handle this well, so I am probably doing something wrong. As a workaround, I used the modadm device list command to see what device names were assigned and then I added the folders to my storage hosts. Run the following command:

 

# mogadm --lib=/usr/local/share/perl/5.8.4 --trackers=192.168.42.1:6001 device list

 

It will list each host and the device name followed by its status and storage available. Here is example output:

 

mogilestorage 1: alive
used(G) free(G) total(G)
dev1: alive 0.892 67.772 68.664

 

This means "mogilestorage" has a host id of "1" and it has one device named "dev1" on it and that device is in the "alive" state (your other statistics will probably be zeros). Using the example output above, you would simply create the directory on mogilestorage.yourdomain.com:

 

# mkdir -p /var/mogdata/dev1

 

Finally, confirm that your devices are configured:

 

# mogadm --lib=/usr/local/share/perl/5.8.4 --trackers=192.168.42.1:6001 device list

 

 

Running MogileFS

 

 

Starting Storage Servers

 

Start each storage server (mogilestorage.yourdomain.com) by running the following command as root:

 

# mogstored --daemon

 

 

Starting Trackers

 

Trackers will not run as root, so you will need to run them as another user. If you created the "mogile" user when setting up the trackers, the following commands will work (assumes you start logged in to mogiletracker.yourdomain.com as root):

 

# su mogile
$ mogilefsd -c /etc/mogilefs/mogilefsd.conf --daemon
$ exit 

 

You can confirm that the trackers are running with the following command:

 

# ps aux | grep mogilefsd

 

If you don't get a list of running processes, the trackers are not running.

 

 

Try It

 

 

Do a Quick Sanity Test

 

The 'mogadm' tool can be used to make sure your trackers are functioning. You need to supply the Perl lib path which has the 'MogileFS.pm' perl module installed – this was installed if you installed the API in the "Setting up the Trackers and Storage Servers" section above. The following example would check all mogile components using the trackers at IP address 192.168.42.1 and 192.168.42.2, both listening on port 6001:

 

# mogadm --lib=/usr/local/share/perl/5.8.4 --trackers=192.168.42.1:6001,192.168.42.2:6001 check 

 

 

Try it with Real Data

 

Create a domain:

 

# mogadm --lib=/usr/local/share/perl/5.8.4 --trackers=192.168.42.1:6001,192.168.42.2:6001 domain add testdomain

 

Add a class to the domain:

 

# mogadm --lib=/usr/local/share/perl/5.8.4 --trackers=192.168.42.1:6001,192.168.42.2:6001 class add testdomain testclass

 

 

Troubleshooting

 

This section is still very incomplete.  Let me know if you have other common problems that need to be added. 

 

When I run mogadm I get "Unable to retrieve host information from tracker(s)":

mogadm requires the tarckers to be running before it is run.

 

When starting the storage daemon I get "ERROR: Directory not found for service mogstored":

You did not create a storage directory or you are starting the mogstored as a user that does not have access to the directory.

 

Problems Connecting:

Make sure your firewall is open. Using the examples in this document, port 7500 and 7501 needed on storage servers, 6001 on trackers.

 

While testing I get “MogileFS::Backend: couldn't connect to mogilefsd backend at /usr/local/share/perl/5.8.4/MogileFS.pm line 56”:

Make sure your tracker connects to the database:

 

# su mogile
$ mogilefsd -c /etc/mogilefs/mogilefsd.conf

 

I get a "REQUEST FAILURE" on "Checking devices..." when doing a mogadm check:

Confirm the devices (folders) exist in the /var/mogdata directory and that the use running the mogstored process has full permissions to these directories. If the device does not exist, add it – it will take a few seconds for mogadm check to reflect the fixed directory. For example, if mogilestorage.yourdomain.com had the device "dev1" on it, you would add the directory:

 

# mkdir -p /var/mogdata/dev1

 

 

Document Revision History

 

2009-10-01 Version 0.06 Chris Kraynik: Many typo fixes, formatting changes.

2007-12-13 Version 0.05 Dormando: Don't scare people over the CVS version.

2006-10-05 Version 0.04 Mike Limon: fixed small typo. (/vag/mogdata)

2006-07-03 Version 0.03 Brett G. Durrett: highlighted need to start trackers before using mogadm (and added troubleshooting reference), formatting changes.

2006-06-26 Version 0.02 Brett G. Durrett: minor typo corrections and fixed characters trashed in converting document to html.

2006-06-25 Version 0.01 Brett G. Durrett: created initial document.