How to install GlusterFS in RHEL/CentOS and Fedora


Introduction to GlusterFS and to install in RHEL/CentOS and Fedora

 

GlusterFS is a distributed file system used in user space. This is a software based file system which accounts to its own flexibility feature.  It provides a wide range of advantages over a centralized file system. Installation of GlusterFS is explained in this article.

 

Merits

  • High performance
  • High availabity
  • Simple
  • Innovative
  • Scale Linearly
  • Elasticity

Gluster outstanding among other distributed file systems

  • Scalable
  • Affordable
  • Open Source
  • Flexible.


Storage concepts in GlusterFS

  • Brick – It is basically any directory to be shared among the trusted storage pool.
  • Trusted Storage Pool – The collection of shared files/directories, which are based on the designed protocol.
  • Block Storage – The devices which the data is being moved across systems in the form of blocks.
  • Cluster – In Red Hat Storage, both cluster and trusted storage pool convey the same meaning of collaboration of servers based on a defined protocol.
  • Distributed File System – A file system in which data is spread over different nodes.
  • FUSE –To create file systems above kernel without involving any of the kernel code.
  • glusterd – glusterd is the GlusterFS management daemon which is the backbone of file system which will be running throughout the whole time whenever the servers are in active state.
  • POSIX –It is the family of standards defined by the IEEE as a solution to the compatibility between Unix-variants in the form of an Application Programmable Interface (API).
  • RAID – It is a technology that gives increased storage reliability through redundancy.
  • Subvolume – A brick after being processed by least at one translator.
  • Translator –It performs the basic actions initiated by the user from the mount point. It connects one or more sub volumes.
  • Volume – It is a logical collection of bricks. All the operations are based on the different types of volumes created by the user.

The different types of volumes

  •  Replicated volume
  • Distributed volume
  • Striped volume
  • Distributed replicated volume.

To install GlusterFS in RHEL/CentOS and Fedora

Requirements


Install CentOS 7 between the two nodes.
Set the host name as “server” and “client“.
Create a working network.
Name the storage disk on both nodes  as “/brick/data“.

Here, we are installing and configuring GlusterFS for the first time for high availability of storage. Now lets use the servers to create volumes and replicate data between them.
Follow the installation process and configuration in both the servers

 

To Enable EPEL and GlusterFS Repository

Before Installing GlusterFS on both the servers, enable EPEL repositories.

[root@server ~]# yum install epel-release -y
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: centos.webwerks.com
 * epel: ftp.cuhk.edu.hk
 * extras: centos.webwerks.com
 * updates: centos.excellmedia.net
Resolving Dependencies
--> Running transaction check
---> Package epel-release.noarch 0:7-5 will be updated
.
.
.
Updated:
  epel-release.noarch 0:7-6                                                                   
Complete!


To enable GlusterFs repository use the following command.

[root@server ~]# wget -P /etc/yum.repos.d http://download.gluster.org/pub/gluster/glusterfs/LATEST/EPEL.repo/glusterfs-epel.repo
--2016-05-10 10:55:22--  http://download.gluster.org/pub/gluster/glusterfs/LATEST/EPEL.repo/glusterfs-epel.repo
Resolving download.gluster.org (download.gluster.org)... 23.253.208.221, 2001:4801:7824:104:be76:4eff:fe10:23d8
Connecting to download.gluster.org (download.gluster.org)|23.253.208.221|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1049 (1.0K)
Saving to: ‘/etc/yum.repos.d/glusterfs-epel.repo’
100%[====================================================>] 1,049       --.-K/s   in 0s      
2016-05-10 10:55:23 (53.1 MB/s) - ‘/etc/yum.repos.d/glusterfs-epel.repo’ saved [1049/1049]

To install GlusterFS

 

Run the following command to install GlusterFS.

[root@server ~]# yum install glusterfs-server -y
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: centos.webwerks.com
 * epel: ftp.cuhk.edu.hk
 * extras: centos.webwerks.com
 * updates: centos.excellmedia.net
Resolving Dependencies
--> Running transaction check
---> Package glusterfs-server.x86_64 0:3.7.11-1.el7 will be installed
.
.
.
Dependency Updated:
  glusterfs-api.x86_64 0:3.7.11-1.el7           glusterfs-libs.x86_64 0:3.7.11-1.el7          
Complete!

 

Start the GlusterFS management daemon

 

Execute the below command to start the GlusterFS services.

[root@server ~]# systemctl start glusterd.service


Check the status of daemon.

[root@server ~]# systemctl status glusterd.service
glusterd.service - GlusterFS, a clustered file-system server
   Loaded: loaded (/usr/lib/systemd/system/glusterd.service; disabled)
   Active: active (running) since Tue 2016-05-10 10:57:32 IST; 8s ago
  Process: 7261 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid --log-level $LOG_LEVEL $GLUSTERD_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 7262 (glusterd)
   CGroup: /system.slice/glusterd.service
           ??7262 /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO
May 10 10:57:32 client systemd[1]: Started GlusterFS, a clustered file-system server.


To Configure a SELinux and iptables

 

Open '/etc/sysconfig/selinux' on both the servers. Edit SELinux permission to “permissive” or “disabled”. Then Save and close the file.

 

[root@server ~]# vim /etc/sysconfig/selinux
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted


Flush the iptables in both nodes.

[root@server ~]# iptables -F


To Configure the Trusted Pool

On "Server"

[root@server ~]# gluster peer probe 192.168.5.190
peer probe: success.


On "Client"

[root@client ~]# gluster peer probe 192.168.5.191
peer probe: success. Host 192.168.5.191 port 24007 already in peer list


Important: Once this pool has been connected, the trusted users alone can probe new servers into this pool.


To check the status of the gluster

On "server"

[root@server ~]# gluster peer status
Number of Peers: 1
Hostname: 192.168.5.190
Uuid: b6c2fd86-af1b-420b-b111-dee1e16d2c3b
State: Peer in Cluster (Connected)


On "client"

[root@client ~]# gluster peer status
Number of Peers: 1
Hostname: 192.168.5.191
Uuid: a4f7053b-35fe-425b-9dab-0b761b32e770
State: Peer in Cluster (Connected)


To list the gluster pool list

On "server"

[root@server ~]# gluster pool list
UUID                    Hostname         State
b6c2fd86-af1b-420b-b111-dee1e16d2c3b    192.168.5.190    Connected 
a4f7053b-35fe-425b-9dab-0b761b32e770    localhost        Connected 


On "Client"

[root@client ~]# gluster pool list
UUID                    Hostname         State
a4f7053b-35fe-425b-9dab-0b761b32e770    192.168.5.191    Connected 
b6c2fd86-af1b-420b-b111-dee1e16d2c3b    localhost        Connected


To Set up a Volume

Create a volume on any single server.

[root@server ~]# gluster volume create dist-volume 192.168.5.190:/brick/data force
volume create: dist-volume: success: please start the volume to access data

 

[root@server ~]# gluster volume start dist-volume
volume start: dist-volume: success


Run the following command to Check the volume info.

[root@server ~]# gluster volume info
Volume Name: dist-volume
Type: Distribute
Volume ID: fd8e2ad9-bb89-4b3d-9e22-743b39f131ad
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: 192.168.5.190:/brick/data
Options Reconfigured:
performance.readdir-ahead: on

 

To Check the volume status of the “server”.


[root@server ~]# gluster volume status
Status of volume: dist-volume
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 192.168.5.190:/brick/data             49152     0          Y       7816 
NFS Server on localhost                     2049      0          Y       4289 
NFS Server on 192.168.5.190                 2049      0          Y       7844 
 
Task Status of Volume dist-volume
------------------------------------------------------------------------------
There are no active volume tasks


To Check volume info in “client”  server


[root@client ~]# gluster volume info
Volume Name: dist-volume
Type: Distribute
Volume ID: fd8e2ad9-bb89-4b3d-9e22-743b39f131ad
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: 192.168.5.190:/brick/data
Options Reconfigured:
performance.readdir-ahead: on


Now verify the volume status in “client” server

[root@client ~]# gluster volume status
Status of volume: dist-volume
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 192.168.5.190:/brick/data             49152     0          Y       7816 
NFS Server on localhost                     2049      0          Y       7844 
NFS Server on 192.168.5.191                 2049      0          Y       4289 
 
Task Status of Volume dist-volume
------------------------------------------------------------------------------
There are no active volume tasks


Important : If volume is not started, the error messages will be stored under ‘/var/log/glusterfs‘ on one or both the servers.

 

To Verify the Volume

Now mount the gluster volume group under the directory “/mnt/gluster”.

[root@client ~]# mkdir /mnt/gluster
[root@client ~]# mount.glusterfs 192.168.5.191:/dist-volume /mnt/gluster
[root@client ~]# df -h
Filesystem                  Size  Used Avail Use% Mounted on
/dev/sda3                    18G  5.1G   13G  30% /
devtmpfs                    486M     0  486M   0% /dev
tmpfs                       494M  140K  494M   1% /dev/shm
tmpfs                       494M  7.1M  487M   2% /run
tmpfs                       494M     0  494M   0% /sys/fs/cgroup
/dev/sda1                   497M  116M  382M  24% /boot
192.168.5.191:/dist-volume   18G  5.1G   13G  30% /mnt/gluster


To make it permanent, mount and add the entry in the fstab file.


#
# /etc/fstab
.
.
.
192.168.5.191:/dist-volume      /mnt/gluster    glusterfs       defaults,_netdev        0 0
[root@client ~]# mount -a


Rebalance – The large amount of data was previously residing, we can perform a rebalance operation to distribute the data among all the bricks including the newly added brick.
Geo-replication – It provides back-ups of data for disaster recovery. Here comes the concept of master and slave volumes. So that if master is down whole of the data can be accessed via slave.
Self-heal – If any of the bricks in a replicated volume are down and users modify the files within the other brick, the automatic self-heal daemon will come into action as soon as the brick is up next time and the transactions occurred during the down time are synced accordingly.


Thank you! for using Linux Help.

You find this tutorial helpful? Share with your friends to keep it alive.
For more help topics browse our website www.linuxhelp.com
Be the first to comment, we value your suggestions. For further queries please comment below.

Tags: Gluster File System
user image Author :  Benjamin