Install “S3Fuse” actually it’s s3fs and mount local disk Amazon EC2


There is a need to mount a S3 bucket to an EC2 server. Please NOTE that mounting drives should not be used in high read write environments. If all you want to do is push a backup to the disk because S3cmd will not do it for you then by all means use this. Do not use it for files that are server via a heavily used web server. (these are not my findings but from a startup company that I was talking to at the 2012 London AWS summit)

The install instructions for fuse are pretty concise so as per usual this will be about those little gotya’s (if that’s how you spell it 🙂 )

The are some pre-requisites as always but these are listed on the site. For S3Fuse installation instructions

To download S3fs

So from the install instructions with some slight amendments of course

yum install gcc libstdc++-devel gcc-c++ fuse fuse-devel curl-devel libxml2-devel 
openssl-devel mailcap make
cd ~
mkdir software
cd software

As usual you need to get the latest stable release and at the time of writing 1.61.1 was stable

NOTE that the install instructions for 1.71 are below

wget http://s3fs.googlecode.com/files/s3fs-1.61.tar.gz
tar xvzf s3fs-1.61.tar.gz
cd s3fs-1.61
./configure --prefix=/usr
make
make install

There is an amendment to this as @ 15/6/2013 there was a new version of 1.71, there we some additional pre-requisites to consider. If you just downloaded this and ran the ./configure command there is a good chance that you would get the following errors.

For the purposes on a search engine and to highlight this error, below can be seen the typical error message when installing s3f3 1.71

checking for DEPS… configure: error: Package requirements (fuse >= 2.8.4 libcurl >= 7.0 libxml-2.0 >= 2.6 libcrypto >= 0.9) were not met:

Requested ‘fuse >= 2.8.4’ but version of fuse is 2.8.3
No package ‘libcrypto’ found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix

OK, the repos that AWS CentOS is hooked up with is not running high enough versions

There are some issues, surrounding the new s3fs version as you can see from error messages

You need to uninstall fuse

You may get the following error

Error in PREUN scriptlet in rpm package realplay
XXXXXXX was supposed to be removed but is not!

rpm -e --noscripts --nodeps fuse
rpm --rebuilddb
yum erase fuse*

We will therefore need to get these and install by hand.

mkdir /opt/software
cd /opt/software
wget http://sourceforge.net/projects/fuse/files/fuse-2.X/2.9.3/fuse-2.9.3.tar.gz
tar -xvzf fuse-2.9.3.tar.gz
cd fuse-2.9.3
./configure --prefix/usr/local
make
make install

OK now to fix some path issue and a final dependancy

export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig

To check if it is installed and the path statement is correct

pkg-config --modversion fuse

This should display the Fuse version

The libcrypto error can be fixed by installing openssl and openssl-devel

yum install -y openssl openssl-devel

OK now to try and install s3f3

/opt/software/s3fs-1.71
./configure --prefix=/usr/local
make
make install
ln -s /usr/local/bin/s3f3 /usr/bin/s3fs

OK to test if you have this installed

s3f3 should then prompt you for a bucket name and credentials

if you get the following error

“s3fs: error while loading shared libraries: libfuse.so.2: cannot open shared object file: No such file or directory”:

issue the following

yum install fuse-libs

You know have 1.71 installed

Irrespective of the version you have installed S3Fuse should now be installed. We need to test this now. So we need to mount a bucket to the server. You will need your Amazon AWS ID and secret. I am assuming that you know how to do this already but if you need a hint for the URL https://aws-portal.amazon.com/gp/aws/securityCredentials

You need to create a .passwd-s3fs file. This is best done as root as it should be stored in the home directory and should of course be secured done.

cd ~
echo accessKeyId:secretAccessKey > .passwd-s3fs
chmod 600 ~/.passwd-s3fs

to test mount a bucket you either need to know the bucket name or use s3cmd to list the bucket names etc… see installing and using s3cmd

Once you know which bucket you want to mount then

cd /mnt
mkdir bucketname - this is only a suggestion but it keeps it consistent and therefore easy to debug

then issue the s3fs commands

/usr/bin/s3fs mybucket /mnt -ouse_cache=/tmp

e.g.

the bucket name is call domainname-website-export so

(all s3 buckets as you know have to be unique and therefore I have adopted the practice of starting every bucket with the domain name)

cd /mnt
mkdir website-export
/usr/bin/s3fs -o allow_other domainname-website-export /mnt/website-export -ouse_cache=/tmp 

NOTE the -o allow – makes the mounted directory accessible by other users of the server.

you should now be able to cd /mnt/website-export and create a file

Now to permanently mount the drive when the server boots up etc… the command for the fstab is as follows :

s3fs#bucketname /mnt/mount_folder fuse allow_other 0 0

e.g.

vi /ect/fstab

vi hint i

s3fs#domainname-website-export /mnt/website-export fuse allow_other 0 0

vi hint Esc :wq

mount -a

You should now have a mounted drive, that is accessible by any user of the server.

Advertisements

2 Comments on “Install “S3Fuse” actually it’s s3fs and mount local disk Amazon EC2”

  1. Findus says:

    Worth pointing out that S3Fuse and S3FS are different
    http://code.google.com/p/s3fs/
    http://code.google.com/p/s3fuse/

    Your instrcutions are for S3FS, not S3Fuse as the title suggests.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s