Setting up GlusterFS on Centos 6.5

Gary Kuhlmann - April 13, 2014

This is a step by step setup on GlusterFS on a single node and drive. Mostly this is a proof of concept but it is also intended to provide an easily scalable method of adding storage space as my needs increase. I don’t need high availability or a very high degree of performance since this will be running in my house.

First, I installed centOS 6.5 got all my updates and mounted my first storage drive, a 4TB Red Drive from Western Digital that is specifically setup for NAS storage. I partitioned it using parted (since fdisk only creates up to 2TB partitions) and formated the partition with an ext4 filesystem.

Now I start the GlusterFS installation:

Add the GlusterFS repo:

wget -P /etc/yum.repos.d http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-epel.repo

Install GlusterFS:

yum -y install glusterfs glusterfs-fuse glusterfs-server

If you run into dependency issues add the epel repo by executing the following:

wget http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
rpm -ivh epel-release-6-8.noarch.rpm

Start GlusterFS and set to start when the system starts:

service glusterd start
chkconfig glusterd on

Now would be the time that each node (or Trusted Storage Pool) would be setup. Since I only have one server, I am skipping this.

Now I can setup a storage volume.

I have already mounted my data drive in /opt/data/drive1. I then create a directory named brick inside. This way, if the drive does not mount for some reason, the brick with error.

mkdir /opt/data/drive1/brick

Create the actual brick in glusterFS:

gluster volume create dist-volume yoda:/opt/data/drive1/brick

Now that the volume is created, we need to start it:

gluster volume start dist-volume

Your glusterFS volume is now mounted. To test I will mount the directory locally:

mkdir /mnt/distributed
mount.glusterfs yoda:/dist-volume /mnt/dristributed

Some additional Commands:

Get volume information:

gluster volume info

Add a brick to an existing volume:

gluster volume add-brick dist-volume yoda:/tmp/brick

Clear a brick to be re-added after it has been removed from a volume:

setfattr -x trusted.glusterfs.volume-id [brick path]
setfattr -x trusted.gfid [brick path]
rm -rf $brick_path/.glusterfs

Source: http://www.sohailriaz.com/glusterfs-howto-on-centos-6-x/
http://www.gluster.org/community/documentation/index.php/HowTos:Brick_naming_conventions