Installing Community Edition Infobright Server


We are a creating our Pentaho environment in the Amazon cloud, we ustilise the Infobright ICE column database for the analyser tool. This allows the software to work extremley fast. The information provide to get this server working is some what limited, and the following is an amalgamation of much research around many of the areas and also snippets from the INSTALL and README files that are supplied. The following instructions were written as I installed and fixed the issues, It should not contain errors as I tried to record everything as I did it. There is always issues with this and I am sure that I have missed some small detail out. Just drop me a line if you spot and error or know of a better way to achieve something. The creation of the server will require a mounted disk to store the database information. Due to the nature of the Amazon this will be a little bit of an experiment. We will create a server with a large EBS volume and a standard server with a large EBS volume attached. We will then bench mark the performance!!!! Good luck and lets get started. Please note unless the instructions say to use the Yum package manger please compile from source, I have had many instances where the package manager has installed the wrong version and software installs just do not work There is an assumption that you know your way arround AWS and the notes that relate to the configuration is where it maybe obscure. You Amazon skills must be around EC2 instance creation including security groups, volume creation and route 53, route 53 is not so important but will make your like a whole lot easier in many circumstances. This will be the server with the Volume mounted Create a Amazon micro instance to start with, will make this a large instance when we are finished, choose a 64 bit version as we may need RAM if the data got large. DO NOT create the 100GB volume when creating the server, I did this and I could not get the disk to format or mount. Create the 100GB volume and attach it to the newly created server. Lauch the server.

fdisk -l

You should get an output some thing like this

Disk /dev/xvda1: 8589 MB, 8589934592 bytes 255 heads, 63 sectors/track, 1044 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000

Disk /dev/xvda1 doesn’t contain a valid partition table

Disk /dev/xvdf: 107.4 GB, 107374182400 bytes 255 heads, 63 sectors/track, 13054 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000

Disk /dev/xvdf doesn’t contain a valid partition table

You are looking for the 100GB disk in this case it is /dev/xvdf

mkdir -p /u10
mkfs -t ext4 /dev/xvdf
vi /etc/fstab

Add this line to the bottom of the file

/dev/xvdf   /u10       ext4    defaults              0 0

The assumption here is that you will mount the disk under u10 To mount the disk

mount -a

To see the size of the disks and the mount state etc..

df

You should get an output something like this

Filesystem           1K-blocks      Used Available Use% Mounted on /dev/xvda1             8361916   1077896   7200160  14% / tmpfs                   304800         0    304800   0% /dev/shm /dev/xvdf            104693300   1673096  97777324   2% /u10

Now for the installation of Infobright You need some prerequisites:

yum install -y gcc libtool zlib-devel perl bison zlib make gcc-c++
 

Create the user and group that will run the Infobright

groupadd mysql
useradd -g mysql mysql

Must of the rest is taken from the README file that you get when you unpack Infobright. If you follow this guide you will not go wrong though. Boost is required for the installation of Infobright

mkdir -p /opt/software/
cd  /opt/software/

Donwload the 1.42.0 version, I tried the 1.50.0 and there was and error relating to the boost_thread when compiling Infobright.

configure: error: Could not link against boost_thread !

wget http://sourceforge.net/projects/boost/files/boost/1.42.0/boost_1_42_0.tar.gz/download
tar -xvzf boost_1_42_0.tar.gz
cd boost*

Review the Boost web site for all of the explanation of the paths etc.. but if you use the paths below you will not go wrong. PLEASE NOTE the – – in front of the prefix

./bootstrap.sh --prefix=/usr/local/boost_1_42_0
 ./bjam install
export BOOST_ROOT=/usr/local/boost_1_42_0

To avoid the following error when compiling Infobright

checking for termcap functions library… configure: error: No curses/termcap library found

cd /opt/software
wget ftp://ftp.gnu.org/gnu/termcap/termcap-1.3.1.tar.gz
tar -xvzf  termcap-1.3.1.tar.gz
cd termcap*
./configure
make
make install

Now to install lsb_release which is needed when installing the last parts of Infobright

wget http://sourceforge.net/projects/lsb/files/lsb_release/1.4/lsb-release-1.4.tar.gz/download
tar -xvzf lsb-release-1.4.tar.gz
cd lsb-release*
make
make install
cd /opt/software/

We would normally get the tar and install it via the instructions below. This though does not work and will error. Miss this chunk out and scroll down a few lines to “Start the install here”

wget https://www.infobright.org/downloads/ice/infobright-4.0.7-0-linux-i686-ice.tar.gz

tar -xvzf infobright-4.0.7-0-*.tar.gz
cp -r infobright-4.0.7-i686 /usr/local

ln -s /usr/local/infobright-4.0.7-i686 /usr/local/infobright
cd /usr/local/infobright
./install-infobright.sh --datadir=/u10/infobright/data \
--cachedir=/u10/infobright/cache --config=/etc/my-ib.cnf --port=5029 \
--socket=/u10/infobright/mysql-ib.sock \
--user=mysql --group=mysql

This will error  This error was received install-infobright.sh: line 234: lsb_release: command not found Installing default databases… Installing MySQL system tables… 130903 12:56:24 [ERROR] Fatal error: Can’t change to run as user ‘mysql’ ; Please check that the user exists! 130903 12:56:24 [ERROR] Aborting 130903 12:56:24 [Note] /usr/local/infobright/bin/mysqld: Shutdown complete Start the install here

wget https://www.infobright.org/downloads/ice/infobright-4.0.7-0-x86_64-ice.rpm

rpm -i infobright-4.0.7-0-x86_64-ice.rpm

wget ftp://rpmfind.net/linux/sourceforge/f/fu/fuduntu/sources/redhat-lsb-4.0-7.fu2012.src.rpm

rpm -Uvh redhat-lsb-4.0-7.fu2012.src.rpm

We now need to download the source and compile this

cd /op/software
wget http://www.infobright.org/downloads/ice/infobright-4.0.7-0-src-ice.tar.gz

tar -xzvf infobright-4.0.7-0-src-ice.tar.gz

cd /opt/software/infobright*

make EDITION=community release 

make EDITION=community install-release 
cp src/build/pkgmt/my-ib.cnf /etc/
cd /usr/local/infobright

When you start Infobright you make encounter this error

error while loading shared libraries: libboost_filesystem.so.1.42.0: cannot open shared object file: No such file or directory

To fix this:

find /. -name libboost_filesystem.so.1.42.0

This will give you a location, on my server

/usr/local/boost_1_42_0/lib

Make sure there is no reference to  /usr/local/boost_1_42_0/lib in your path statement echo $PATH Create a file called boost and add the path statement found above

vi /etc/ld.so.conf.d/boost.conf

/usr/local/boost_1_42_0/lib

Write the file and then

ldconfig

To make sure it has worked

ldconfig -p | grep libboost_filesystem.so

When starting a service the mysqld-ib needs to access lsb init-functions I am installing on a 64 bit Amazon (red-hat based) so

wget ftp://rpmfind.net/linux/sourceforge/f/fu/fuduntu/yum/2012/STABLE/RPMS/redhat-lsb-4.0-5.fu14.x86_64.rpm

To find your version if rpm based http://rpmfind.net/linux/rpm2html/search.php?query=redhat-lsb or search for your distributions rpm There are some prerequisites.

 yum install -y gettext mailx patch

Now we need to create mysql-ib etc.. This is done by copying some files from the source into the installation directory and then running and install script. To avoid this error when installing “error while loading shared libraries: libboost_filesystem.so.1.42.0: cannot open shared object file: No such file or directory” make sure you do the LD_LIBRARY_PATH element of the instructions When copying the files just ensure that you overwrite all of them when prompted

export PATH=$PATH:/usr/local/bin
cp -r /opt/software/infobright-4.0.7/build/community/release/vendor/support-files .
cp -r /opt/software/infobright-4.0.7/src/build/pkgmt/*.in support-files/
cp -r /opt/software/infobright-4.0.7/build/community/release/vendor/scripts/ .
cp -r /opt/software/infobright-4.0.7/src/build/pkgmt/confman.sh .
cp -r /opt/software/infobright-4.0.7/src/build/pkgmt/install-infobright-linux.sh .
install-infobright-linux.sh

Now we need to change the mysqld-ib file so that when starting libboost_filesystem.so.1.42.0 can be found.

cd /etc/init.d
vi mysqld-ib

Find the line that look like this

PATH=/sbin:/usr/sbin:/bin:/usr/bin:$basedir/bin

Copy this line and change it to this

PATH=$PATH:$basedir/bin

This will protect you path statement if you have added more entries in ./bashrc Now add the following lines after the export PATH statement

NEWLDPATH=/usr/local/boost_1_42_0/lib
LDP=$LD_LIBRARY_PATH
if [ "${LDP}" == "" ]; then
LD_LIBRARY_PATH+=$NEWLDPATH
else
if [ -d "$NEWLDPATH" ] && [[ ! $PATH =~ (^|:)$1(:|$) ]]; then
 LD_LIBRARY_PATH+=:$NEWLDPATH
fi
fi
export LD_LIBRARY_PATH

Save the file I needed the normal version of mysql also on the server, this is needed so I can insert and update the tables, which will then be exported into the infobright tables.

yum install mysql-server mysql

Because I am using the faster disk I want to change the data directory for mysql

service mysqld stop
vi /etc/my.cnf

Add the following line

datadir = /u10/mysql

Now copy the data directory to the new location

cp -R /var/lib/mysql /u10
rm -r -f /var/lib/mysql
chown -R mysql:mysql /u10/mysql
vi /etc/selinux/config

set the line SELINUX=enforcing to SELINUX=disabled

to just disable selinux so you do not have to reboot

echo 0 >/selinux/enforce
service mysqld start

To make sure that this works

service mysqld-ib start

Now we have it working we need to secure the database with passwords and removing the test database etc.. Because I have installed mysql you need to do a temporary work around, or change the scripts.

service mysqld stop

mv /usr/bin/mysql  /usr/bin/mysql_org

ln -s /usr/local/infobright/bin/mysql /usr/bin/mysql

/usr/local/infobright/bin/mysql_secure_installation

service mysqld-ib stop

rm -f /usr/bin/mysql

mv /usr/bin/mysql_org /usr/bin/mysql

This can be used if you installed the rpm and then copied in the compiled elements of the code

./postconfig.sh

Infobright post configuration ————————————– Using postconfig you can: ————————————– (1) Move existing data directory to other location, (2) Move existing cache directory to other location, (3) Configure server socket, (4) Configure server port, (5) Relocate datadir path to an existing data directory.

Please type ‘y’ for option that you want or press ctrl+c for exit.

Current configuration:

————————————– Current config file: [/etc/my-ib.cnf] Current brighthouse.ini file: [/usr/local/infobright-4.0.7-x86_64/data/brighthouse.ini] Current datadir: [/usr/local/infobright-4.0.7-x86_64/data] Current CacheFolder in brighthouse.ini file: [/usr/local/infobright/cache] Current socket: [/tmp/mysql-ib.sock] Current port: [5029] ————————————–

(1) Do you want to copy current datadir [/usr/local/infobright-4.0.7-x86_64/data] to a new location? [y/n]:y Give new datadir path (e.g. /opt/datadirnewpath/data):/u10/infobright/data (2) Option to change CacheFolder is disabled when option 1 is chosen! (3) Do you want to change current socket [/tmp/mysql-ib.sock]? [y/n]:y Give new socket::/u10/infobright/mysql-ib.sock (4) Do you want to change current port [5029]? [y/n]:n (5) Relocation is disabled when options 1-4 are chosen!

————————————– Datadir(/usr/local/infobright-4.0.7-x86_64/data) is going to be copied to /u10/infobright/data New socket is going to be :/u10/infobright/mysql-ib.sock ————————————–

Please confirm to proceed? [y/n]:y Copying /usr/local/infobright-4.0.7-x86_64/data to /u10/infobright/data …is done. You can now remove/backup your old /usr/local/infobright-4.0.7-x86_64/data … Done!

./postconfig.sh

(1) Do you want to copy current datadir [/usr/local/infobright-4.0.7-x86_64/data] to a new location? [y/n]:n

(2) Do you want to move current CacheFolder [/usr/local/infobright-4.0.7-x86_64/cache] to a new location? [y/n]:y

Give new CacheFolder path:/u10/infobright/cache

Please confirm to proceed? [y/n]:y

(1) Do you want to copy current datadir [/u10/infobright/data] to a new location? [y/n]:n

(2) Do you want to move current CacheFolder [/u10/infobright/cache] to a new location? [y/n]:n

(3) Do you want to change current socket [/tmp/mysql-ib.sock]? [y/n]:n

Please confirm to proceed? [y/n]:y

cd /usr/local/infobright/data
rm -f -r  *

To avoid the following error when starting the database

mysqld_safe mysqld from pid file /u10/infobright/data/ip-10-226-137-80.pid ended 120723 21:42:03 mysqld_safe Starting mysqld daemon with databases from /u10/infobright/data 120723 21:42:03 [Warning] options –log-slow-admin-statements, –log-queries-not-using-indexes and –log-slow-slave-statements have no effect if –log_slow_queries is not set 120723 21:42:03 [ERROR] Fatal error: Can’t change to run as user ‘mysql’ ; Please check that the user exists!

120723 21:42:03 [ERROR] Aborting 120723 21:42:03 [Note] /usr/local/infobright-4.0.7-x86_64/bin/mysqld: Shutdown complete

chown -R mysql:mysql /usr/local/infobright-4.0.7
chown -R mysql:mysql /usr/local/infobright-4.0.7-i686 
chown -R mysql:mysql /usr/local/infobright 
chown -R mysql:mysql /u10/infobright
cd /usr/local/infobright-4.0.7-x86_64

We now have a working server. Not much use if you can not connect to it 🙂 The first thing is to allow external access to the database

vi /etc/my-ib.cnf

add these two lines

skip-host-cache
skip-name-resolve

Your file should look something like this # The MySQL server [mysqld] skip-host-cache skip-name-resolve You also need to ensure that skip_networking is commented, your files should look like. #skip-networking #server-id=1 # log-bin=mysql-bin So to get on and set the security so that you can access it from an external server you set it up just like a MYSQL server.

mysql-ib -u root -p

This will get you a sql prompt. You will use the normal mysql to create and grant options.

create database userdatabase;
create user 'user'@'%' identified by 'userpassword';
grant all privileges on userdatabase.* to 'user'@'%' with grant option;
Advertisements

2 Comments on “Installing Community Edition Infobright Server”

  1. Or M. says:

    Hi,
    great article! helped me alot when configuring an EC2 instance with IB.
    my notes:
    1. the EC2 should be Red-Hat if we want to follow your instructions more easily (i.e. fdisk -l does nothing on Amazon AMI)

    2. two missing installs:
    yum install python-devel bzip2-devel
    are necessary before building boost

    thanks!
    Or m.

  2. Hi Or m.

    I suspect when you launched the Amazon AMI you tried fdisk -l using the ec2-user if you sudo su then try it, it should work.

    The reason for not using the Red Hat AMI is the license costs.

    Hope you got IB working fully for you.

    Cheers

    Al


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s