Sharing Raspberry Pi Disk With Mac

Here is how to share your raspberry pi disk whether internal or external USB device with your Mac book

We are going to use is the Netatalk

http://netatalk.sourceforge.net

For all the commands you can either use sudo at the front or as I prefer which is not recommended just switch to root

So

sudo su

Step 1: Installation

From a terminal window on your PI

apt-get update
apt-get install netatalk

Step 2: Configuration

Stop the service:

systemctl stop  netatalk

Open the configuration file:

vi /etc/netatalk/AppleVolumes.default

You can edit the mount folder if you wish or leave the default value:

~/ “Home Directory”

You will find these settings at the bottom of the file

Start the service again

systemctl start  netatalk

Step 3: Open the Finder

Open the Finder on your mac and you will be able to see the raspberry pi with a few seconds in the Navigation Pane.

Use the button “Connect As…” at the right top of the Finder.

The default user is pi and the password is raspberry

Advertisements

Installing Plexmedia Server on a Raspberry Pi

Ok there are many instructions that help you install Plexmediaserver on a Raspberry Pi, unfortunately there is one missing component when using the newest version of raspbian.

I used the following site for the start of the config. Thepi.io

A direct lift from the site.

You as always can use sudo in front of all the commands, I hate typing so I use sudo su to start.

Step 1: Check that you have the HTTPS transport package installed

We’re going to use the HTTPS transport package in this process. It should be automatically installed on Raspbian, but that isn’t true for some older versions. Stay in the command line and type:

apt-get install apt-transport-https

This will install the latest version. If you already have it, you’ll just get a message kicked back telling you so. Either way, you’re now ready for the next step.

 

Step 2: Add the dev2day repository to your package source list

We need the dev2day repository because it contains Plex. First, we need a crypt O key for the dev2day website.

wget -O – https://dev2day.de/pms/dev2day-pms.gpg.key | sudo apt-key add –

Next, we’ll add dev2day’s repository to the package source list.

echo “deb https://dev2day.de/pms/ jessie main” | sudo tee /etc/apt/sources.list.d/pms.list

Finally, update the package list.

apt-get update

 

Step 3: Download Plex

Yep, we’re finally at this stage!

apt-get install -t jessie plexmediaserver

You’ll have to hit “y” when prompted to approve the download.

 

Step 4: Permissions chores

This is where many of the instructions are a little out of date 😦

“Let’s set Plex to run under the Pi user.

sudo nano /etc/default/plexmediaserver

Your terminal window will fill with a big text document. Look for the line that says PLEX_MEDIA_SERVER_USER=plex. Delete “plex” and type in “pi,” so that the line reads PLEX_MEDIA_SERVER_USER=pi.

Now hit CTRL+X to exit. Type Y for “yes” to save your changes and hit Enter to stick with the same old file name. Then, in the terminal, type:

sudo service plexmediaserver restart”

The principle is correct, you want to run the plexmediaserver as “pi” this allows you to access mounted folders etc..

So to achieve this follow these simple steps.

systemctl stop plexmediaserver
nano /lib/systemd/system/plexmediaserver.service

The listing will look something like this

[Unit]

Description=Plex Media Server for Linux

After=network.target

[Service]

Environment=”PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR=/var/lib/plexmediaserver/Library/Application Support”

Environment=PLEX_MEDIA_SERVER_HOME=/usr/lib/plexmediaserver

Environment=PLEX_MEDIA_SERVER_MAX_PLUGIN_PROCS=6

Environment=PLEX_MEDIA_SERVER_TMPDIR=/tmp

ExecStartPre=/bin/sh -c ‘/usr/bin/test -d “${PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR}” || /bin/mkdir -p “${PLEX_MEDIA_SERVER_APPLIC$

ExecStart=/bin/sh -c ‘LD_LIBRARY_PATH=/usr/lib/plexmediaserver “/usr/lib/plexmediaserver/Plex Media Server”‘

Type=simple

User=plex

Group=plex

Restart=on-failure

RestartSec=5

StartLimitInterval=60s

StartLimitBurst=3

[Install]

WantedBy=multi-user.target

 

You are looking for the user and group element, you need to change these to pi

User=pi

Group=pi

Save the file.

Now you need to change the permissions on the plex server itself.

This is the important part 🙂

chown -R pi:pi /var/lib/plexmediaserver
systemctl start plexmediaserver

Step 5: IP address chores

You’ll want your Raspberry Pi Plex server to have a static IP address, so that you only have to connect once on each client device. First, get your IP:

hostname -I

This will kick back your IP address. Copy it or write it down. Let’s open your command line txt file.

nano /boot/cmdline.txt

At the bottom of the command line text file, type ip= followed by your IP address. Save and exit the file (CTRL+X, then Y, then Enter).

 

Step 6: Restart your Pi

Let’s restart. You can do this right from the command line with

reboot

 

Step 9: Add files to your server

Now we’ve showed you how to setup a Raspberry Pi Plex server – but we haven’t showed you how to put anything on it. Let’s start populating that server.

First, make sure you’ve got your files attached to your Pi somehow – like through a USB external hard drive, for instance.

Then, open your browser and type your IP address followed by :32400/web/

You can now take it from here and create an account etc…


Get a delimited list in Oracle SQL

There are many times that I need to retrieve a delimited list in SQL and every time I have to track the solution down.

 

So here it is.

SELECT RTRIM (
 rtrim(XMLAGG (XMLELEMENT (e, column_name || ',')).EXTRACT ('//text()'), ',') name
 FROM table_name;

 

Just to break this down a little

rtrim(XMLELEMENT (e, column_name || 'This is the delimiter')).EXTRACT 
('//text()'), 'This is the delimiter')




Converting PDFs’ into text

PDFtoTXT

I am currently working on a project to extract the text from a PDF purchase order and create an xml file to feed into our ERP system, fully automated obviously.

I therefore needed a way to extract the text from a PDF at the command line, luckily there is a cool utility called poppler-utils. This works perfectly and is extremely fast.

I have installed this on both a CentOS server and more recently and Ubuntu server,

The caveat to this product is that you need the PDF to be real and not an image, this is why the Ubuntu server may be used.

mkdir ~/software
wget https://poppler.freedesktop.org/poppler-0.40.0.tar.xz
wget https://poppler.freedesktop.org/poppler-data-0.4.7.tar.gz
tar xvf poppler-0.40.0.tar.xz
tar xvf poppler-data-0.4.7.tar.gz

There are a few requirements

yum -y install fontconfig fontconfig-devel
yum -y install cairo cairo-devel libjpeg libjpeg-devel libcurl-devel gtk-doc 
yum -y install libtool gcc-c++ lcms2 openjpeg-libs xz openjpeg-devel
yum -y install libtiff-devel lcms2-devel

cd poppler-data-0.4.7
make install

tar (child): xz: Cannot exec: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now

Ensure that you install xz

cd poppler-0.40.0
./configure

ERROR

checking for a BSD-compatible install… /usr/bin/install -c
checking whether build environment is sane… yes
checking for a thread-safe mkdir -p… /bin/mkdir -p
checking for gawk… gawk
checking whether make sets $(MAKE)… yes
checking whether make supports nested variables… yes
checking whether make supports nested variables… (cached) yes
checking for style of include used by make… GNU
checking for gcc… no
checking for cc… no
checking for cl.exe… no
configure: error: in `/root/software/poppler-0.40.0′:
configure: error: no acceptable C compiler found in $PATH
See `config.log’ for more details

ERROR

No package ‘fontconfig’ found

Ensure that you installed  fontconfig fontconfig-devel

INFORMATION

Building poppler with support for:
font configuration: fontconfig
splash output: yes
cairo output: no (requires cairo >= 1.10.0)
qt4 wrapper: no
qt5 wrapper: no
glib wrapper: no (requires cairo output)
introspection: no
cpp wrapper: yes
use gtk-doc: no
use libjpeg: no
use libpng: no
use libtiff: no
use zlib: no
use libcurl: no
use libopenjpeg: no
use cms: no
command line utils: yes
test data dir: /root/software/poppler-0.40.0/./../test

Warning: Using libjpeg is recommended. The internal DCT decoder is unmaintained.
Warning: Using libopenjpeg is recommended. The internal JPX decoder is unmaintained.

Ensure that you installed  cairo cairo-devel libjpeg libjpeg-devel libcurl-devel gtk-doc

make
make all-recursive

make[1]: Entering directory `/root/software/poppler-0.40.0′
Making all in goo
make[2]: Entering directory `/root/software/poppler-0.40.0/goo’
CXX libgoo_la-gfile.lo
../libtool: line 1129: g++: command not found
make[2]: *** [libgoo_la-gfile.lo] Error 1
make[2]: Leaving directory `/root/software/poppler-0.40.0/goo’
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/root/software/poppler-0.40.0′
make: *** [all] Error 2

Ensure that you  installed libtool gcc-c++ lcms-libs openjpeg-libs

make install

You should now have a working copy of the software.

To convert a pdf you just simply issue

pdftotext -layout PDF_Name  Output_Filename

 

 

Ubuntu Install

mkdir ~/software

apt-get -y install libcurl4-gnutls-dev libcairo2-dev libcairo2 libjpeg libjpeg-dev \
libtiff5-dev libgtk-doc g++ fontconfig fontconfig-dev fontconfig*

wget https://launchpad.net/ubuntu/+archive/primary/+files/poppler-data_0.4.6.orig.tar.gz
wget https://launchpad.net/ubuntu/+archive/primary/+files/poppler_0.41.0.orig.tar.xz

tar -xvf poppler-data_0.4.6.orig.tar.gz

tar -xvf poppler_0.41.0.orig.tar.xz

cd 

cd poppler-data-0.4.6/

make install

cd poppler-0.41.0/

./configure –prefix=/usr

make

make install

 


Installing s3fs on Centos

centos-logos3_amazon

Many years ago I install s3fs on Centos servers and wrote about it, today I needed to install it on a new server.

I went straight to the instructions and of course as in everything Open Source they were out of date. So here for the next few months are the new install instructions. 🙂

cd ~
mkdir software
cd software
wget -O master.zip https://github.com/s3fs-fuse/s3fs-fuse/archive/master.zip 

Some prerequisites 

yum -y install automake libcurl gcc-c++ \
libcurl-devel libxml2 libxml2-devel libtool gettext gettext-devel \
openssl openssl-devel
 
unzip master.zip
cd s3fs-fuse-master
./autogen.sh 

 

ERROR

— Make commit hash file ——-
— Finished commit hash file —
— Start autotools ————-
./autogen.sh: 38: ./autogen.sh: aclocal: not found
— Finished autotools ———-

Ensure that you have installed automake

./configure --prefix=/usr

ERROR

checking whether the C++ compiler works… no
configure: error: in `/root/software/s3fs-fuse-master’:
configure: error: C++ compiler cannot create executables
See `config.log’ for more details

Ensure that you have installed  gcc-c++

No package ‘fuse’ found
No package ‘libcurl’ found
No package ‘libxml-2.0’ found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables common_lib_checking_CFLAGS
and common_lib_checking_LIBS to avoid the need to call pkg-config.

Ensure that you installed fuse-devel  libcurl-devel libxml2-devel

ERROR

configure: error: Package requirements (fuse >= 2.8.4 libcurl >= 7.0 libxml-2.0 >= 2.6) were not met:

Requested ‘fuse >= 2.8.4’ but version of fuse is 2.8.3

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables common_lib_checking_CFLAGS
and common_lib_checking_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.

You need to uninstall fuse

You may get the following error

Error in PREUN scriptlet in rpm package realplay
XXXXXXX was supposed to be removed but is not!

rpm -e --noscripts --nodeps fuse
rpm --rebuilddb
yum erase fuse*

cd ~/software
wget -O fuse-2_9_bugfix.zip https://github.com/libfuse/\
libfuse/archive/fuse-2_9_bugfix.zip

unzip fuse-2_9_bugfix.zip
cd libfuse-fuse-2_9_bugfix

I tried the install with version 3, this was a disaster

wget -O libfuse.zip https://github.com/libfuse/libfuse/archive/master.zip
unzip libfuse.zip
cd libfuse-master/

./makeconf.sh

Running libtoolize...
./makeconf.sh: line 4: libtoolize: command not found
config.rpath not found! - is gettext installed?

Ensure that libtool gettext gettext-devel

./configure --prefix=/usr/local
make
make install

OK now to fix some path issue and a final dependency

export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig

To check if it is installed and the path statement is correct

pkg-config –modversion fuse

This is from version 3

pkg-config --modversion fuse3
ln -s /usr/local/lib/pkgconfig/fuse3.pc /usr/local/lib/pkgconfig/fuse.pc

cd ~/software/s3fs-fuse-master/
./configure --prefix=/usr

ERROR

checking for common_lib_checking… configure: error: Package requirements (fuse >= 2.8.4 libcurl >= 7.0 libxml-2.0 >= 2.6) were not met:

No package ‘fuse’ found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables common_lib_checking_CFLAGS
and common_lib_checking_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.

ERROR

checking for DEPS… configure: error: Package requirements (fuse >= 2.8.4 libcurl >= 7.0 libxml-2.0 >= 2.6 libcrypto >= 0.9) were not met:

No package ‘libcrypto’ found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables DEPS_CFLAGS
and DEPS_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.

Ensure that  openssl openssl-devel

make
make install

OK to test if you have this installed

s3f3 

should then prompt you for a bucket name and credentials

s3fs: missing BUCKET argument.
Usage: s3fs BUCKET:[PATH] MOUNTPOINT [OPTION]…

ERROR

s3fs: error while loading shared libraries: libfuse.so.2: cannot open shared object file: No such file or directory

yum install fuse-libs

You need to create a .passwd-s3fs file. This is best done as root as it should be stored in the home directory and should of course be secured done.

cd ~
echo accessKeyId:secretAccessKey > .passwd-s3fs
chmod 600 ~/.passwd-s3fs

Now create a mount point for the bucket

cd /mnt
mkdir bucketname - this is only a suggestion but it keeps it 
consistent and therefore easy to debug

then issue the s3fs commands (to text if the mount works)

s3fs mybucket /path/to/mountpoint -o passwd_file=~/.passwd-s3fs
 

NOTE the -o allow – makes the mounted directory accessible by other users of the server.

If you encounter any errors, enable debug output:

s3fs mybucket /path/to/mountpoint -o passwd_file=~/.passwd-s3fs -d -d -f -o 
f2 -o curldbg

Now to permanently mount the drive when the server boots up etc… the command for the fstab is as follows :

s3fs#bucketname /mnt/mount_folder fuse allow_other 0 0

e.g.

vi /ect/fstab

s3fs#domainname-website-export /mnt/website-export fuse _netdev,allow_other 0 0

To mount the bucket

mount -a 


Install s3fs Ubuntu 14.04 LTS

ubuntu-logo112s3_amazon

Many years ago I install s3fs on Centos servers and wrote about it, today I needed to install in an Ubuntu server.

I went straight to the instructions and of course as in everything Open Source they were out of date. So here for the next few months are the new install instructions. 🙂

cd ~
mkdir software
cd software
 wget https://github.com/s3fs-fuse/s3fs-fuse/archive/master.zip

Some prerequisites 

apt-get -y install automake build-essential libfuse-dev fuse libcurl3
libcurl3-dev libxml2 libxml2-dev
unzip master.zip
cd s3fs-fuse-master
 ./autogen.sh 

ERROR

— Make commit hash file ——-
— Finished commit hash file —
— Start autotools ————-
./autogen.sh: 38: ./autogen.sh: aclocal: not found
— Finished autotools ———-

Ensure that you have installed automake

./configure --prefix=/usr

ERROR

checking whether the C++ compiler works… no
configure: error: in `/root/software/s3fs-fuse-master’:
configure: error: C++ compiler cannot create executables
See `config.log’ for more details

Ensure that you have installed  build-essential

No package ‘fuse’ found
No package ‘libcurl’ found
No package ‘libxml-2.0’ found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables common_lib_checking_CFLAGS
and common_lib_checking_LIBS to avoid the need to call pkg-config.

Ensure that you installed libfuse-dev  libcurl3-dev libxml2-dev

A quick hint, if you are looking for a package e.g. libcurl you could use the following command :-

apt-cache search libcurl

make
make install

 

OK to test if you have this installed

s3f3 should then prompt you for a bucket name and credentials

s3fs: missing BUCKET argument.
Usage: s3fs BUCKET:[PATH] MOUNTPOINT [OPTION]…

You need to create a .passwd-s3fs file. This is best done as root as it should be stored in the home directory and should of course be secured done.

cd ~
echo accessKeyId:secretAccessKey > .passwd-s3fs
chmod 600 ~/.passwd-s3fs

Now create a mount point for the bucket

cd /mnt
mkdir bucketname - this is only a suggestion but it keeps it 
consistent and therefore easy to debug

then issue the s3fs commands (to text if the mount works)

s3fs mybucket /path/to/mountpoint -o passwd_file=~/.passwd-s3fs
 

NOTE the -o allow – makes the mounted directory accessible by other users of the server.

If you encounter any errors, enable debug output:

s3fs mybucket /path/to/mountpoint -o passwd_file=~/.passwd-s3fs -d -d -f -o 
f2 -o curldbg

 

Now to permanently mount the drive when the server boots up etc… the command for the fstab is as follows :

s3fs#bucketname /mnt/mount_folder fuse allow_other 0 0

e.g.

vi /ect/fstab

s3fs#domainname-website-export /mnt/website-export fuse _netdev,allow_other 0 0

Allowing access to a mounted drive from a non root user is a bit of a headache.

change the /etc/fuse.conf file and un-comment the user_allow_other

# Allow non-root users to specify the allow_other or allow_root mount options.
user_allow_other

Then add the mount line to /etc/fstab

s3fs#bucketname mount_point  fuse _netdev,allow_other,umask=700,use_rrs  0 0

The highlighted areas are the imported entries

To mount the bucket

mount -a 

 

 

 

 


Technical Bits n Pieces

This really is and aide memoire, a scrap pad, random commands etc ……. as I come across errors, workarounds, fixes.

This is all about making those notes so that I don’t have to trawl Google again to find the answers and at least I know these commands work 🙂

How to grep IP addresses in a file?

I needed to tail a file for IP addresses as some thing was using a large amount of web server resource. The log file is called httpd-access

 tail -f httpd-access_log | grep -E -o “([0-9]{1,3}[\.]){3}[0-9]{1,3}”

How to deny and ip address from accessing your server?

Yes you can use a firewall, but if you are behind a load balancer or proxy server then you can not.

In either the httpd.conf or the .htaccess file add the following lines :-

httpd.conf – add these to the directory element of the file

Order allow,deny
Allow from all
deny from Enter IP address

At the end of the file

<Files 403.shtml>
order allow,deny
allow from all

.htaccess – add this to the IfModule element, You can deny IP addresses or use a RewriteCond and Rule.

# Block Bad Bots
RewriteCond %{HTTP_USER_AGENT} ^.*(Vagabondo).*$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*(AhrefsBot).*$ [NC]
# ISSUE 403 / SERVE ERRORDOCUMENT
RewriteRule . - [F,L]
order allow,deny
allow from all

deny from Enter Ip address here

Add this at the end of the file

<Files 403.shtml>
order allow,deny
allow from all
</Files>

The server will redirect traffic to a 403 error.

 

How to log the source IP address when behind a load balancer or proxy?

You use a load balancer to spread the load of your web sites, but how can you find the ip address of who is using your site, this is specific to the Apache server but something similar will work on nginx I am sure.

In the httpd.conf file change the

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

to

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"%{X-Forwarded-For}i\"" combined

You now should create some custom logs files, again in httpd.conf add the following lines to the virtualhost element

ErrorLog logs/httpd-error_log
CustomLog logs/httpd-access_log combined

…..

Allowing SSH access to Ubuntu 14.0 LTS Desktop

We are running some software that needs a Linux desktop, I am not a big fan of Ubuntu but it is one of the listed supported platforms.

I obviously need vnc access, wow what a pain. Centos / Redhat easy to do Ubuntu !!!!

So I though ssh on and just configure, well there is no SSH access out of the box. Build for the GUI generation. To get ssh access

sudo apt-get update
sudo apt-get install openssh-server

 

You now have ssh access!!!

 

Create a patch file and then the application of the patch

If you need to create a differential file for redeployment on another system then creating a patch file is ideal.

You will need the patch utilities installed

yum install patch

Now to create your patch file. Copy the original file into the same directory

e.g. cp config.php config.php_chg

Now update the config.php_chg file with all of your amendments

Now to create the file

diff -Naur config.php config.php_chg > config.php.patch

If you want to now apply the patch

patch config.php < config.php.patch

You now should keep the file in a safe place or if it is not a sensitive change, i.e. containing passwords then you can add it to https://gist.github.com/