Emoncms on DietPi


I am a huge fan of DietPi – lightweight and fast, runs an a large number of SBCs. The aim here was to build a local EmonCMS server as simply as possible. Doing it even from a Rasbian Stretch minimal image can be a bind – DietPi makes some of the tricky bits as easy as selecting from a menu. Not everything can be done that way and some command line effort is required. I started with a fresh V6.16 DietPi install. Well actually it is a base VM image I have that I simply updated. Yes, DietPi runs very happily on a VM, is much quicker than on a Pi and no issues with SD Cards corrupting. In addition, you can take snapshots so less risk of ruining an installation!

The other aspect is that these instructions will install a LLAP stack of Lightppd, PHP7 and MariaDB. Personally I think Apache is rather bloated.

Setting up


The initial setting up is best done via the DietPi interface. Simply type dietpi-config to enter the menu system.

  1. DietPi-config – Advance options – Time sync mode – time sync daemon + drift (more reliable time)
  2. DietPi-config – Security options – change root password and hostname (optional)
  3. DietPi-config – Network Options: Adapters  – fixed IP and turn off IPV6 (optional)


DietPi has a menu system for installing a number of different software packages. You can either do these installs via the menu system or shortcut it by using the command line syntax dietpi-software install [ID] (except for the SSH change).

  1. Avahi daemon [152] – gives you a hostname on your network.
  2. Install GIT [17]
  3. Install LLAP [82] – It will suggest you do not need this but go ahead anyway.
  4. DietPi-Software – SSH Server – OpenSSH (Optional – allows WinSCP to transfer files – replaces defualt dropbear)
  5. Install redis-server [91] – Only required if you are going to use Redis, but it is suggested you do!
  6. Install Mosquitto [123] – Only required if you intend to run an MQTT broker on this machine (I run the broker on my HASSIO instance). Not needed to publish and subscribe to topics.

Command line installs

Next you need to install a few bits directly. NOTE in DietPi you do most things as root (you can setup other users, I never bother) so sudo is not required.

  1. apt-get -y install php7.0-dev – This gives you the phpenmod and phpdismod commands
  2. apt-get -y install build-essential – Is used by PECL to build compile the redis and mqtt php modules.
  3. apt-get -y install php-pear
  4. apt-get -y install mosquitto-clients – Optional, required if you want to debug MQTT.
  5. pecl channel-update pecl.php.net
  6. pecl install redis-3.1.6 – This is the PHP extension for REDIS – the version is important.
  7. printf "extension=redis.so" | sudo tee /etc/php/7.0/mods-available/redis.ini 1>&2
  8. phpenmod redis – Enable the PHP Redis extension.

MQTT [edit]

I suddenly realised that I had not included how to install MQTT. The simplest way is to follow this guide (remembering you do not need sudo).

Configure Lighttpd

nano /etc/lighttpd/lighttpd.conf – Edit the Lighttpd configuration:

Add in mod_rewrite

And add these lines at the end…

Either restart Lighttpd or reboot the server (I like to do the latter) systemctl restart lighttpd


I was going to refer to the main documentation for this, but it is somewhat out of date and inconsistent so…

The Base System

  1. cd /var/www
  2. git clone -b master https://github.com/emoncms/emoncms.git You may want to use the stable branch instead.

Setup The Database

  1. mysql -e "DELETE FROM mysql.user WHERE User='root' AND Host NOT IN ('localhost', '', '::1'); DELETE FROM mysql.user WHERE User=''; DROP DATABASE IF EXISTS test; DELETE FROM mysql.db WHERE Db='test' OR Db='test_%'; FLUSH PRIVILEGES;"
  2. sudo mysql -e "CREATE DATABASE emoncms DEFAULT CHARACTER SET utf8;"
  3. mysql -e "CREATE USER 'emoncms'@'localhost' IDENTIFIED BY 'emonadmin'; GRANT ALL ON emoncms.* TO 'emoncms'@'localhost'; FLUSH PRIVILEGES;"

Create Data Directories

  1. mkdir /var/lib/{phpfiwa,phpfina,phptimeseries}
  2. chown www-data:root /var/lib/{phpfiwa,phpfina,phptimeseries}
  3. chmod 755 /var/lib/{phpfiwa,phpfina,phptimeseries}

Setup Log file

  1. touch /var/log/emoncms.log
  2. chmod 666 /var/log/emoncms.log
  3. chown root:www-data /var/log/emoncms.log

Edit EmonCMS Settings

  1. cd /var/www/emoncms && cp default.settings.php settings.php
  2. nano /var/www/emoncms/settings.php

First run

Open a browser window to [YourIpAddress]/emoncms. At the user prompt click on Register. Register a user then log in. First stop is the administration page and check database. Everything should now be ready to go!

You may wish to add in some additional Modules.

You may also wish to enable MQTT for subscribing and publishing.

Facebooktwitterby feather
Posted in DietPi, OpenEnergyMonitor, Technical - General | 2 Comments

Else is never necessary – Really?

I had an interesting experience with Codacy today.  I got this error.

When doing my IT Degree (many moons ago) I was taught 2 things (amongst others you understand):

  1. An If statement always has an else. You should always handle a possible situation even if it should never happen.
  2. There should only ever be one exit point from a function, loop etc.

Perhaps these mantras stemmed from a safety critical standpoint (I was using Ada).

However, it seems the ‘modern way’ doesn’t agree. Well tough, I still think I am right and any other way is just building in problems.

Facebooktwitterby feather
Posted in Coding and Programming | Leave a comment

Keeping time on a Pi

Again, this blog fall into the category of a ‘I need to remember how I did this’ post.  I actually started writing this months ago but never finished it.  Good job as some of what I originally said was wrong and we have moved on from Jessie to Stretch.

This started when I accidentally discovered that the date/time on one of my Pi’s was significantly out.  I won’t bore you with the things that I tried and failed, rather just tell you how to fix it.

You can check the date&time your own system by simply typing date and seeing if it is right or not.

Finding out how to get the right time set is remarkably difficult.  In the end I discovered that you can simply use this command (reference);

sudo sntp -s time.google.com

NTP is the protocol that synchronises time across the internet.  There are a number of free to use servers that you can query to get the current date & time.

One important fact is that if your date & time are significantly out, you cannot use ntp to set the time correctly.  Set it manually so about the right time then sync.

On more recent versions/distributions of Linux,  timedatectl  may be used to query and change the system clock and its settings.  Note this is not a daemon in it’s own right.  On Linux there are 2 daemons you can use, ntpd  or systemd-timesyncd . However you must not have both enabled at the same time. ntpd is the older daemon. systemd-timesyncd is the newer one and is a builtin service.

Now have a look at the daemon that controls synchronising your system with an ntp server.

The best explanation of this daemon I found was here, with another guide here.

As you can see from the output above, NTP is enabled and synchronised and my timezone is set correctly, it is likely that in the response you see, they are not.  Firstly, if your timezone is not right I’d suggest using sudo raspi-config to set it correctly.

In order to use ntp to keep your clock right, you first need to point the daemon to a time server.  The recommendation is to use a suitable one out of the ntp pool project; http://www.pool.ntp.org/en/use.html . To do so you need to edit the file  sudo nano /etc/systemd/timesyncd.conf  so that your Time section looks something like this;

Interestingly, the guide above suggested the line should start ‘NTP’ but as all I did to the standard line was uncomment it and change debian to uk.

To now switch on the synchronising use the command sudo timedatectl set-ntp true

One final thing, if you are using UFW you may find you need to open the port with  sudo ufw allow 123/udp

You may also need to restart the services  sudo systemctl restart systemd-timesyncd.service

Your system should now keep time 🙂


Facebooktwitterby feather
Posted in Technical - General | Leave a comment

Old DietPi Orange Pi Images

If, like me, you are a fan of DietPi, you may well have been irritated by recent events.  The decision to move to a completely new image that could not be upgraded to, was understandable, if unfortunate.  These things happen to any OS with Windows being the obvious comparison.

Unfortunately this upgrade coincided with a spat with armbian which seemed to revolve around a school yard ‘my image is better than yours’ Twitter argument.  The end result is that DietPi have dropped links with Armbian and so there are no new images for the OrangePi family which Fordee readily admits were some of the most popular downloads.

Fortunately, I had (unusually) quickly upgraded to the last Jessie version (v158) on my OrangePi Zero+ before it all went pete tong.  Although no new images are being made available, it is not as if there is no further support in that you can still do the apt-get update/upgrade so it is only the added support DietPi brought that has been lost.  DietPi did host some specific packages in their repositories so that will become an issue as time goes on.

DietPi are also being robust and deleting links to old images out there that are posted on the forum.  One user has kindly provided a download of the images, the link for which I got before it was removed.  If you want the last Jessie OrangePi image from DietPi you can get them here.


Facebooktwitterby feather
Posted in DietPi, OrangePi | 2 Comments

Adding a DietPi VM to an EXSi Host

This blog very much falls into the ‘how in the hell did I do that’ category.

A bit of background.  I have had an HP MicroServer for a number of years just sitting there not doing a lot except running an XP VM with a mailserver on it called Mailtraq.  This was a hangover from when mail servers were pretty poor and GoogleMail was a twinkle in someone’s eye.  It also helped with dial on demand (who remembers that!), NNTP and Demon Internet.  Suffice to say life has moved on so the XP VM needed to be binned and so, sadly, did the Mailtraq installation.

What then to do with the MicroServer.  It runs a free version of VMWare’s EXSi server (5.x) which I happily connect to via the old vSphere client.  Whilst this is a little old hat, I know it and can find my way around it so ‘it ain’t broke, don’t fix’ is the mantra.  I then realised that DietPi have some VM images so I thought I’d give them a go.  Whilst running things on RaspberyPis seems like a good idea, for somethings it really isn’t.  The network speed is poor (shared USB Bus), SD Card wear and a lack of grunt.  The DietPi images are built for the VMware Workstation but I was able to convert them using the VMware vCenter standalone converter with little problem.  The speed of the application was fast compared to a B+ Pi so all was well and good.

DietPi have just released a new platform.  This has caused some angst as the older platform is now simply not supported.  However, it is a step change for them, so I picked up the new V6 image and tried to convert it.  No joy. I kept getting an error “Unable to Obtain Hardware information for the selected machine”.  At first I thought it was an error with the new image file but on trying an image I had previously converted successfully, it also failed.

Following some searching, I am sure that the problem is with security on Windows 10.  I do not know if it is MS or BitDefender.  I had a brief foray with BitDefender but it interfered with Bash on Windows (no network) and they were unwilling to investigate without me doing all the work so it go canned. Getting rid of all the BitDefender garbage seems a bit tricky!  I did try opening the converter as Administrator, disabling UAC and disabling MS Defender all to no avail.

So plan B was to upload the VMDK and VMX files.  I was sure this had failed before but I had a go.  First boot failed on OS type.  This was fixed by selecting ‘otherLinux-64’.

Next boot did not like number of CPUs.  Changed that in setting.

Next boot did not like the disk type.  Error message “Failed to open disk scsi0:0: Unsupported or invalid disk type 7”.  The issue here is that the disk type is for VMware Workstation.  However, fixing this was relatively simple.  I had SSH enabled on the EXSi host so SSH in and use  vmkfstools -i <sourceVM> <targetVM> (KB item) to clone the disk to the right format.

The VM now booted and DietPi started.  One final issue was the network did not want to configure.  I fixed this by deleting the network adapter and installing a new one.

Having now got a DietPi VM that has run once, I will clone this and create the various VMs I need. http://www.mustbegeek.com/create-copy-of-existing-virtual-machine-in-esxi-server/

Facebooktwitterby feather
Posted in DietPi, Linux | Leave a comment

Displaying buttons on the TFT

Having successfully hooked my small TFT up to the Raspberry Pi, I now wanted to use it! As it was a touch screen version what I wanted was buttons.

This is the end result;

TFT initial state

TFT display updated after a button press

The code is to be found in this gist:

A few comments on the false starts and things I learned (code is well commented I think):

  • I tried to install pygameUI but this failed because of the way I had implemented the TFT itself.
  • pygame is structured in a way I was unfamiliar with (fairly new to python). Key concepts are:
    • The screen (the area you update)
    • Rect – an object that defines a rectangular area of the screen.  I created an array of Rect objects then gave each a centre.  Once defined they will always display at that location unless you change it.
    • You output various layers then update the display.
    • Updating the display is best done on the rects that have changed although in this case, frame rate is not an issue.
    • I used centres as I found that trying to create a border using a top left reference did not result in a constant thickness of the border all round.

I now need to find a way to mount the TFT permanently.  I also really would like an enclosure – I’m going to hunt down a 3D one or pay a visit to my local hacklab.

I also need to expand my code to pull instructions off and send instructions to the main IOT server I have to control the heating.



Facebooktwitterby feather
Posted in Coding and Programming, Python | Tagged , , | Leave a comment

Forcing npm to Update Packages to Latest

I have had a problem getting npm to update packages to the latest version.  The packages involved are packages that have been installed directly and have no dependencies.  It it is down to the use of semantic versioning in npm / node.js but knowing that does not help!

I have a package, canvas-gauges, installed via npm, that I wanted to update.  Running

told me that;

I wanted to update it to the latest version but despite repeated searches, on how to do this, I could not force npm to update the package.

Trying these did not work;

I also tried installing and running ncu but that complains there is no package.json file.

There is nothing wrong in what npm is doing, it is following the ‘Wanted’ version even though I wanted the latest evrsion.

Ultimately the solution was simple (just not easily found);



Facebooktwitterby feather
Posted in Linux, Node-RED | Tagged | Leave a comment

Using a Small TFT with a Raspberry Pi


As I have said before part of the point of this blog is so I have a record of how I did things and this definitely fits into that category.  If others find it useful, that is a bonus.

I have had a small 2.4″ TFT with touch sitting around for ages.  I looked at using it when it arrived, but compling a kernel put me right off.  So this week the notion took me to try again and I discovered it was actually very easy with newer versions of Raspbian.

[Edit 16 Dec 17] I had made an error in the table below putting both clocks on GPIO08. I have had some PCBs made up to simplify the connection – the error blew the TFT I was using sadly.


There are a couple of key sources I used and in places I have blatantly plagiarized them (they are that good) so as to bring it all together, and in case the source disappears (it happens).

  1. Wiring and OS tasks
  2. tslib info
  3. fbTFT info
  4. getting started with pygame
  5. More on pygame
  6. Fixing Coordinates on Jessie


The background detail, found in the first reference, is interesting. It seems the old notro type method is being phased out in preference for overlays.  Worth reading.


I was fortunate in that my board was very similar to the board in reference 1 so I could use those mappings for my wiring.  If you have a different board then head over there and it is explained how to work out the wiring for your specific overlay.

Panel Pin Function name GPIO Pin No.
T_IRQ Panel Touch Interrupt GPIO25 22
T_DO Touch Data Out GPIO09 21
T_DIN Touch Data In GPIO10 19
T_CS Touch Chip Select SPI_CE1_N GPIO07 26
T_CLK Touch Clock GPIO11 23
LED Backlight GPIO18– see note 12
D/C TFT Data / Command GPIO24 18
RESET Reset GPIO23 16
CS TFT Chip Select SPI_CE0_N GPIO08 24
GND Ground GND
VCC 3V3 supply

I did the wiring using a breadboard as a couple of the pins need the same GPIO pin.  Not pretty but functional!

Note the original sources comment on wiring up the LED Backlight.  For now I have just connected to the 3.3V.

Setting up the OS

As an aside, although I did this initially on a clean install of Raspbian Jessie, I then moved to DietPi. The main reason for this is that you can create a configuration file for such things as WiFi SSID & Password and a fixed IP address.  I could then do everything headless from the go.

The OS changes themselves are simplicity itself.

Edit the file /boot/config.txt (on DietPi you need to edit /DietPi/config.txt) and include the following lines;

And reboot the Pi. That’s it.  If it is OK, the screen will light up during boot, then go blank.

Next step is to check and calibrate the touchscreen.  I deviate a bit from my main source as there were 2 extra packages to install (references 2 & 3).

I found I needed to do the calibration before I did the test so (remembering to change the event number and fb if necessary);

and then

If all is well, you can drag the cross hairs or draw in the screen.

You now need to check the touch input. to do this;

and then run evtest  If you are running headless, the touch input is likely to be event0, if you are not, again head over to the first reference and see how to interpret the contents of  /dev/events/

The AdaFruit tutorial on this has some instructions for creating a udev rule to map the touch screen events to a fixed point.  I have not managed to get this to work and I suspect it is because I am using overlays.

pygame and TSLIB problems with Jessie

I won’t bore you with how I got here, suffice to say I realised that the coordinates of the touch input were not correctly reported by the OS to pygame. After a bit of digging I found reference 6.  In summary, to fix the issue create and run the following script;

This fixed the coordinates for me.

Python and TFT

The aim of all this is to have a small touchscreen that I can use to provide simple control of my heating and hot water – basically a boost function.  I’ll blog about that when I get a bit further.  In the meantime, the following code will test out the touch responses.


Edit 5 June 2017

I have now sorted out some more useful code to display buttons which you will find here.




Facebooktwitterby feather
Posted in Hardware, Linux, RaspberryPi | Tagged , , | Leave a comment

Bash History

On several occasions, I have lost some of my bash history, as the ssh session has not been exited cleanly. This annoyed me as I use the history as a means to remember what I have done.  It prompted me to investigate if I could fix it and Unix being Unix, of course you can.

The History part of my .bashrc file now looks like this;

I tried adding a timestamp but that just looked messy so I took it out again.


Facebooktwitterby feather
Posted in Linux | Tagged , | Leave a comment

Installing pywws and running it as a Daemon

I have been using pywws for a number of years to download the data from my weather station and it simply just works.  There is good support from the author though there are some gaps in the documentation.  Moving pywws to the new Pi gave me the opportunity to update the package which had not been done for a while (ain’t broke don’t fix).

The weather station is a WH1080 compatible system from Maplin.  It is often on offer for about £60 and you can buy spares for all the external parts including the transmitter at a very reasonable price which is a major bonus.

Installing pywws via pip is relatively painless but what I would urge, is that you check out the dependencies first and get them installed before the main package.  It is easy to miss or skip over this step.  There are 2 options for USB libraries and I plumped for the libsub pip install libusb1 .

One of the beauties of this weather station and pywws combination is that the weather station itself stores data so you can disconnect the USB from the base station to play and not lose any data as it just catches up once reconnected.

Once installed, I plugged in the weather station to the new pi to check that it was communicating – I had to do the changes suggested as it would only work as sudo otherwise. Doing this test also created some of the system files.

I then pulled across all the data files using rsync plus a couple of system files I had modified.  These modifications allowed pywws to communicate to EmonCMS and to put data out on mqtt. The way this is setup is a little convoluted (to my mind), not that well documented, and it had taken me a while previously to work it out. To do it you need to create a new ‘service’.

New pywws service

To create a new pywws service you need to create 2 files in the service folder(/usr/local/lib/python2.7/dist-packages/pywws/services), a service file and a template.  The template is used to create the format of the message you wish to send and the service file defines some of the parameters that the template may use.  By defining the parameters correctly, you can also change these in the main configuration file weather.ini in the data folder.

I have created a GitHub repository for the new service files for EmonCMS.  I am using the http API for pushing the data to EmonCMS. One of the problems I had in setting this up is that EmonCMS does not accept valid JSON so it took a bit of time to get the template right.

Running pywws live-logger as a Daemon

Pywws has a daemon version of the main logging script using python-daemon. I have never felt completely comfortable with this as it seems to me that by following the instructions, you can end up with a daemon starting a daemon. The original install was Wheezy based so I had created a file for init.d to use (available from the GitHub repository) but on the newer Jessie based system, I thought I’d try a systemd setup.

After reading this excellent guide on systemd, I generated the following service file and placed it in the systemd folder (/etc/systemd/system).  After a couple of minor teething problems, I got it running and it has just worked since.  This seems a better solution.

Monitoring pywws

On the old Wheezy system, I used Monit to monitor both the daemon and a specific folder.  For a while, pywws had a habit of just stopping the writing of data.  The daemon was still running just no data appeared. I therefore set Monit up to monitor a specific folder to check it was modifed regularly.  As I have installed Monit I brought this across (also on the GitHub repository).

However, one of the advantages of systemd is that it has built-in monitor and restart functionality so Monit cannot be used to monitor the daemon itself.

Final Steps

Once I was happy that the system was working fine, I plugged the weather station back into the original Pi, restarted it and left it running for a while to catch up.  I then stopped the daemon, used rsync to copy across the files again (I blatted that data on the new Pi first just to be sure) and plugged the station into the new Pi.  A quick reboot and it all seems very happy.

All that remained was to set up some web access but that is another blog.

Facebooktwitterby feather
Posted in Technical - General | Leave a comment