☰
✕
✕

How to combine several partitions together to form single larger Storage using Mhddfs

To combine several partitions together to form single larger Storage using Mhddfs

This article is about how to combine smaller partitions into a larger virtual storage using Mhddfs. There is a two good solution for this. First solution is, create a RAID array of disk. RAID contains usable disk space and storage reliability. Another solution is termed as mhddfs. Mhddfs is a driver that forms small files into a big virtual file system. This is an easy way for storing a large amount of data.

Features of Mhddfs

  • File system and System Information needs to get.
  • File system attributes to be set.
  • Files and Directories is to create, Read, Remove and write.
  • File locks and Hard links on single device is to be supported.

Install Mhddfs in Linux

Dnf package manager is mentioned below we may use it for Fedora 22+ systems.

dnf installmhddfs

On Debian distributions you can install mhddfs package using the following command.

apt-get update & &  apt-get install mhddfs

To install in CentOS/Fedora systems

Before installing the mhddfs driver, you need to install epel repository and some dependency packages for mhddfs driver.
To install Epel-repository for Centos and use the below command.

[root@linuxhelpDesktop]# yum install epel-release
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
...
...
  Installing : epel-release-6-8.noarch                                      1/1 
  Verifying  : epel-release-6-8.noarch                                      1/1 
Installed:
  epel-release.noarch 0:6-8                                                     
Complete!

To install the dependency packages fuse-devel,gcc,gcc-c++, glibc-devel,uthash-devel use the following commands.

[root@linuxhelp Desktop]# yum install fuse-devel
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
...
...
Installed:
  fuse-devel.x86_64 0:2.8.3-4.el6                                               
Complete!

[root@linuxhelp Desktop]# yum install gcc gcc-c++
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
...
...
Installed:
  gcc.x86_64 0:4.4.7-16.el6            gcc-c++.x86_64 0:4.4.7-16.el6           
Dependency Installed:
  glibc-devel.x86_64 0:2.12-1.166.el6_7.7                                       
Complete!

[root@linuxhelp Desktop]# yum install glibc-devel
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
 * base: centos.webwerks.com
 * epel: epel.mirror.net.in
 * extras: centos.webwerks.com
 * updates: centos.webwerks.com
Setting up Install Process
Package glibc-devel-2.12-1.166.el6_7.7.x86_64 already installed and latest version
Nothing to do

[root@linuxhelp Desktop]# yum install uthash-devel
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
...
...
  Installing : uthash-devel-1.9.9-6.el6.noarch                              1/1 
  Verifying  : uthash-devel-1.9.9-6.el6.noarch                              1/1 
Installed:
  uthash-devel.noarch 0:1.9.9-6.el6                                             
Complete!

To install mhddfs in CentOS

First download the mhddfs package using the following wget command.

[root@linuxhelp Desktop]# wget http://mhddfs.uvw.ru/downloads/mhddfs_0.1.39.tar.gz
--2016-03-28 19:11:42--  http://mhddfs.uvw.ru/downloads/mhddfs_0.1.39.tar.gz
Resolving mhddfs.uvw.ru... 178.79.155.71
...
...
100%[======================================> ] 40,665       200K/s   in 0.2s    
2016-03-28 19:11:48 (200 KB/s) - â &euro &oelig mhddfs_0.1.39.tar.gzâ &euro  saved [40665/40665]

To extract the downloaded package using the following command.

[root@linuxhelp Desktop]# tar -xvf mhddfs_0.1.39.tar.gz 
mhddfs-0.1.39/
mhddfs-0.1.39/debian/
mhddfs-0.1.39/debian/source/
...
...
mhddfs-0.1.39/Makefile
mhddfs-0.1.39/README
mhddfs-0.1.39/mhddfs.spec

To run the make command in the extracted directory

[root@linuxhelp Desktop]# cd mhddfs-0.1.39/
[root@linuxhelp mhddfs-0.1.39]# make
mkdir -p obj
touch obj/obj-stamp
gcc -Wall -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse   -DFUSE_USE_VERSION=26 -MMD -c src/debug.c -o obj/debug.o
gcc -Wall -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse   -DFUSE_USE_VERSION=26 -MMD -c src/flist.c -o obj/flist.o
...
...
gcc -Wall -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse   -DFUSE_USE_VERSION=26 -MMD -c src/usage.c -o obj/usage.o
gcc -Wall -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse   -DFUSE_USE_VERSION=26 -MMD obj/debug.o obj/flist.o obj/main.o obj/parse_options.o obj/tools.o obj/usage.o -o mhddfs -pthread -L/lib64 -lfuse -lrt -ldl  

[root@linuxhelp mhddfs-0.1.39]# ll
total 148
-rw-rw-r--. 1 1000 1000  1864 Jun 17  2012 ChangeLog
-rw-rw-r--. 1 1000 1000   710 Jun 17  2012 COPYING
drwxrwxr-x. 3 1000 1000  4096 Jun 17  2012 debian
-rw-rw-r--. 1 1000 1000 35147 Jun 17  2012 LICENSE
-rw-rw-r--. 1 1000 1000  5447 Jun 17  2012 Makefile
-rwxr-xr-x. 1 root root 51403 Mar 28 19:16 mhddfs
-rw-rw-r--. 1 1000 1000  2635 Jun 17  2012 mhddfs.1
-rw-rw-r--. 1 1000 1000  3773 Jun 17  2012 mhddfs.spec
drwxrwxr-x. 2 1000 1000  4096 Jun 17  2012 mnt
drwxr-xr-x. 2 root root  4096 Mar 28 19:16 obj
-rw-rw-r--. 1 1000 1000  4311 Jun 17  2012 README
-rw-rw-r--. 1 1000 1000  6623 Jun 17  2012 README.ru.UTF-8
drwxrwxr-x. 3 1000 1000  4096 Jun 17  2012 src
drwxrwxr-x. 2 1000 1000  4096 Jun 17  2012 tests

Now copy the mhddfs file to /usr/bin and /usr/local/bin directories.

[root@linuxhelp mhddfs-0.1.39]# cp mhddfs /usr/bin
[root@linuxhelp mhddfs-0.1.39]# cp mhddfs /usr/local/bin

Mhddfs

Step 1:
Currently mounted HDD.

[root@linuxhelp mhddfs-0.1.39]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2              15G  3.9G  9.9G  28% /
tmpfs                 495M  228K  495M   1% /dev/shm
/dev/sda1             194M   29M  156M  16% /boot
/dev/sda5             2.0G   67M  1.9G   4% /new
/dev/sda6            1020M   34M  935M   4% /test

Mount Point name needs to be point out. It will be used by later.

Step: 2
Create /mnt/virtual_storage for mounting the virtual storage.

[root@linuxhelp ~]# mkdir /mnt/virtual_storage

Step: 3
Use the following command to mount the virtual storage.

[root@linuxhelp ~]# mhddfs /new,/,/test /mnt/virtual_storage -o allow_other
mhddfs: directory ' /new'  added to list
mhddfs: directory ' /'  added to list
mhddfs: directory ' /test'  added to list
mhddfs: mount to: /mnt/virtual_storage
mhddfs: move size limit 4294967296 bytes

-o allow_other option is used to make virtual file system visible to all users not only the user who created it.

Step: 4
Now open the /etc/fstab file and add the following line to mount the virtual storage permanently. You need to be a root user to open this file.

mhddfs#/new,/,/test     /mnt/virtual_storage     fuse     defaults    0 0

To restart the system,

[root@linuxhelp~]#init 6

Step: 5
To see all the File systems run df -h.

[root@linuxhelp ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2              15G  3.9G  9.9G  28% /
tmpfs                 495M  228K  495M   1% /dev/shm
/dev/sda1             194M   29M  156M  16% /boot
/dev/sda5             2.0G   67M  1.9G   4% /new
/dev/sda6            1020M   34M  935M   4% /test
/new / /test           18G  4.0G   13G  24% /mnt/virtual_storage

Step: 6
To unmount the virtual storage use the following command.

[root@linuxhelp Desktop]# umount /mnt/virtual_hdd
Tag : Mhddfs
FAQ
Q
Are there any mhddfs write performance issue or low powered CPU?
A
yes, I don't use it anymore as it reduces, ... I noticed that mhddfs consume a lot of CPU (over 50%) in 4 threads.
Q
Where to create a log file for a mhddfs?
A
Specify a file that will contain debug information.
path= logfile=/path/to/file.log
Q
What is the option to find the free space in mhddfs?
A
This option accepts suffixes:
[mM] - megabytes
[gG] - gigabytes
[kK] - kilobytes
For an information about the additional options see an output of:
mhddfs -h
Q
How do I unmount mhddfs based fuse file systems?
A
Use the umount command to detaches the /virtual.data/ file system:
Syntax: "umount /virtual.data/"
Q
What is a command to create new mount point directory and how to join in mhddfs?
A
To create mount point

Syntax:"mkdir /virtual.data"

To join use this command

Syntax: "mhddfs /disk1,/disk2,/disk3 /virtual.data -o allow_other"