Mobile broadband with Raspberry Pi on Three UK & getting a public IP.

If you’ve seen my previous post, you’ll know I was trying to get a decent backup of my Raspberry Pi which I have managed to do.

The next step was to get it to be able to stream a video over HTTP – I tried zoneminder but it’s just too much for the Pi. So instead I used mjpg_streamer, much more lightweight and from what I’ve read seems to use the GPU of the Pi.

Anyway the trickiest bit of setting up the Pi was trying to get the Wifi working on startup and getting the broadband to connect on startup. So I thought I’d share my config files for you guys to see.

For Wifi my config is as below. Although most links on the web say you need to configure a wpa_supplicant.conf, for me this caused issues (I’m unsure as to the cause) but this config worked for me. Just replace YourSSID with your networks name and NetworkPass with your network password.

The key thing here is the allow-hotplug and the auto wlan0, it tries to set it up as soon as it is available but doesn’t throw an error message.

auto lo

iface lo inet loopback
iface eth0 inet dhcp

auto wlan0
allow-hotplug wlan0
iface wlan0 inet dhcp
wpa-ssid "YourSSID"
wpa-psk "NetworkPass"
#wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
#iface default inet dhcp

Now onto the more tricky topic of setting up the 3G modem, mine is a Huawei E160 with USB ID 12d1:140c. First thing is first, just make sure you have usb-modeswitch installed and it is working (in newer versions of Linux this just works). And also install wvdial:

apt-get install wvdial

Now edit the /etc/wvdial.conf to the same the following:

[Dialer Defaults]
Init1 = ATZ
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Stupid Mode = 1
Modem Type = Analog Modem
ISDN = 0
Phone = *99#
Username = three
Password = three
Modem = /dev/ttyUSB0
Dial Command = ATDT
Baud = 9600
check DNS = no
auto DNS = no


[Dialer three]
Init2 = ATZ
Init3 = ATE0 V1 &D2 &C1 S0=0 +IFC=2,2
Init5 = AT+CGDCONT=1,"IP","3internet"
ISDN = 0
Modem = /dev/ttyUSB0
Modem Type = Analog Modem
Baud = 460800
check DNS = no
Auto DNS = no

This important thing to note here is that there are TWO profiles setup above, the default profile and the three profile. Note that if you use the three.co.uk as the APN you get a NAT IP address (i.e. you are part of the Three UK internal network, so don’t have a public IP – exactly like your home network where you are behind your router), but if you use the 3internet APN then you get a public IP! This important if you want to access your Pi from the outside world!

So fire up wvdial and check it all works:

wvdial three

This is all well and good, but pointless if I wanted my Pi to be just plug and play (i.e. start mobile broadband connection, start mjpg_streamer and email me the IP address!)

The easiest way which allows me to control this is to simply have a crontask which calls three other scripts. The other three scripts are 1) start wvdial, 2) start mjpg_streamer and 3) email me the new IP address

Wrapping commands inside 3 separate until loops meant if anything should die it tries to restart it! This is really important when you don’t have physical access to the Pi – at least you can SSH onto it! (see this stackoverflow answer here: )

I think thats about it! Drop a comment below if anything is unclear!

Using DD to dump only the partition data from a USB/SD Card on the Raspberry Pi

So I was playing recently with my Pi and wanted to backup the SD card – the issue with the commands given on the raspbian website is that if you DD an image to a larger SD card (say 16GB) and then make a backup of it, you will get a 16GB image, so for raspbian this is 2.9G of useful stuff with the rest being random useless data.

Well there is a way to avoid this and have a very nice correctly sized image. The gist of this is that we determine the block where the partition ends and dump everything until that point. This works just as well for any storage device.

Now the raspbian image is exactly 3276800000 bytes; when you DD this to a SD card and run fdisk against it you get this:


linux-suse:/home/rm # fdisk -l

Disk /dev/mmcblk0: 15.9 GB, 15931539456 bytes, 31116288 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00090806

Device Boot Start End Blocks Id System
/dev/mmcblk0p1 8192 122879 57344 c W95 FAT32 (LBA)
/dev/mmcblk0p2 122880 6399999 3138560 83 Linux

A simple calculation here of 6399999 x 512 = 3276799488, which is only 1 block (512 bytes) less than the size of the image (3276800000).

So in order to make sure I get a correct image with all the data we use the count and bs (block size) parameters of DD. The total number of blocks DD will write is count x bs so in order to write 3276800000 bytes I chose a count of 4096 and a bs of 800000. So the full DD command would be:

linux-suse:/home/rm # dd if=/dev/mmcblk0 of=raspbian_2014_09_13.img bs=800000 count=4096
4096+0 records in
4096+0 records out
3276800000 bytes (3.3 GB) copied, 112.452 s, 29.1 MB/s

My reason for choosing a block size of 800000? Well because I have plenty of RAM and reading lots of data in large chunks is much faster (29MB/s is pretty good!).

Setting up PostgreSQL 9.3 on CentOS 7

So I had some problems with setting up PostgreSQL on CentOS and getting a remote user working properly. Once you have it installed (install instructions on the site – also note that the service postgresql-9.3 initdb command doesn’t work on CentOS 7 just call it manually, see the PostgreSQL wiki page).

I’ll assume you have a clean PostgreSQL database and you’ve just installed it using YUM.

– Create a new user, and give it a password.

createuser -P openfire

– Edit the pg_hba.conf file and change the listen address to:

listen_addresses = '*'

– And edit the postgresql.conf file and add the following line:

host openfire openfire 0.0.0.0/0 password

For me it will be over local VMware network so password in plaintext (as the password option above implies) will be fine.

– Reboot PostgreSQL:

systemctl restart postgresql-9.3

– Finally try logging in (remember that if you don’t specify the hostname by default the Peer connection method will be used – and you may get an error)

psql -h 172.16.145.135 -U openfire -W

Custom database with Openfire 3.9.3 (PostgreSQL 9.3)

OK so the custom database integration with Openfire is not quite as straight forward as you are led to believe. But here is the full text of my openfire.xml so you can adapt it to your requirements.

What Openfire actually does is parse the XML file as though it was a properties file, each node is a separated by a dot. For example the driver node is database.defaultProvider.driver, you can see all this in the DefaultDatabaseConnectionProvide.java as the documentation doesn’t detail what properties are actually available.

Also note that the username and password are now encrypted in the openfire.xml, however when I created this file they were in plain text – without the encrypted attribute, so just remove this attribute and add your user/pass, Openfire will replace them with encrypted equivalents.

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <!--
  3.    This file stores bootstrap properties needed by Openfire.
  4.    Property names must be in the format: "prop.name.is.blah=value"
  5.    That will be stored as:
  6.        <prop>
  7.            <name>
  8.                <is>
  9.                    <blah>value</blah>
  10.                </is>
  11.            </name>
  12.        </prop>
  13.  
  14.  
  15.    Most properties are stored in the Openfire database. A
  16.    property viewer and editor is included in the admin console.
  17. -->
  18. <!-- root element, all properties must be under this element -->
  19.  
  20. <jive>
  21.   <adminConsole>
  22.     <!-- Disable either port by setting the value to -1 -->  
  23.     <port>9090</port>  
  24.     <securePort>9091</securePort>
  25.   </adminConsole>  
  26.   <locale>en</locale>  
  27.   <!-- Network settings. By default, Openfire will bind to all network interfaces.
  28.      Alternatively, you can specify a specific network interfaces that the server
  29.      will listen on. For example, 127.0.0.1. This setting is generally only useful
  30.       on multi-homed servers. -->  
  31.   <!--
  32.    <network>
  33.        <interface></interface>
  34.    </network>
  35.    -->  
  36.   <connectionProvider>
  37.     <className>org.jivesoftware.database.DefaultConnectionProvider</className>
  38.   </connectionProvider>  
  39.   <database>
  40.     <defaultProvider>
  41.       <driver>org.postgresql.Driver</driver>  
  42.       <serverURL>jdbc:postgresql://postgres:5432/openfire</serverURL>  
  43.       <username encrypted="true">3b1b74e293878b7f66acd3672f718c8d4f755c29688261567ba61b373c1f6959</username>
  44.       <password encrypted="true">e9ece47242052450d1b59446c3c4cf9667c011802a8a403c69a8890b1fbdff33</password>
  45.       <testSQL>select 1</testSQL>
  46.       <testBeforeUse>false</testBeforeUse>
  47.       <testAfterUse>false</testAfterUse>
  48.       <minConnections>5</minConnections>
  49.       <maxConnections>25</maxConnections>
  50.       <connectionTimeout>1.0</connectionTimeout>
  51.     </defaultProvider>
  52.   </database>
  53.   <setup>true</setup>
  54.   <jdbcProvider>
  55.     <driver>org.postgresql.Driver</driver>
  56.     <connectionString>jdbc:postgresql://postgres:5432/openfire</connectionString>
  57.   </jdbcProvider>
  58. </jive>

Installing Openfire 3.9.3 on a clean CentOS 7 image.

Installing Openfire is pretty straightforward the simple RPM does pretty much all of the work for you. But here is a quick guide for those of you that have been struggling.

Download openfire:

wget http://www.igniterealtime.org/downloads/download-landing.jsp?file=openfire/openfire-3.9.3-1.i386.rpm

Run the following command

rpm -Uhv openfire-3.9.3-1.i386.rpm

If you don’t have java installed you’ll get the following error

Job for openfire.service failed. See ‘systemctl status openfire.service’ and ‘journalctl -xn’ for details.

A look at the log files will reveal no jre is installed:

less /opt/openfire/logs/nohup.out

nohup: failed to run command ‘/opt/openfire/jre/bin/java’: No such file or directory

Install Server JRE on your system

mkdir /usr/java

mv server-jre-8u20-linux-x64.tar.gz /usr/java

cd /usr/java

tar -zxvf server-jre-8u20-linux-x64.tar.gz

ln -s jdk1.8.0_20/ latest

vi /etc/profile.d/java.sh

Add the following to the java.sh file:

export JAVA_HOME=/usr/java/latest
export PATH=${JAVA_HOME}/bin:${PATH}

Then press escape type

:wq

and press enter

Openfire still will not start as we installed from the RPM and it includes its own JRE.

So edit the openfire file:

vi /etc/init.d/openfire

and scroll down and replace

OPENFIRE_RUN_CMD=”${JAVACMD}…

with

OPENFIRE_RUN_CMD=”/usr/java/latest/bin/java…

And thats openfire should start:

systemctl start openfire.service

Tigase vs Openfire for building a instant messaging startup

Having spent a little time reading up about the XMPP protocol (and also taking notice how quite a few instant messaging providers are being bought out for a few billion dollars!) I decided I needed to do something similar; it would shore up my skill set further and if users really like it then even better so win win for me.

So initially I took a look at Tigase, what attracted me to it was that it was open source and the performance was good (they had conducted tests to prove it). I also had taken a look at Openfire at the time but discarded it because the general consensus was that Tigase had the better performance. Fast forward about a year and now Tigase is AGPL (meaning you need to also open source your code if you use it – even if it’s only provided over the Internet), so Openfire is really the only choice left (it’s Apache License. You could however obtain the old Tigase GPL code and use that.

But why choose one of these versus many others? Well these were the only two that had a complete implementation, used enterprise languages and had good documentation.

Over the next few weeks/months I’ll be trying to add more content to this blog.