Having a cluster built out of Raspberry Pi’s is not only fun but also eases your work. As we discussed in our latest article in the series you can use the cluster to compile software or make it gather data from various sources on the Internet. There are many uses for such a thing.
However, after compiling for an hour straight you might get curious as of how your nodes perform. How they work under load, if they’re not under-powered or if the CPU temperature is not rising above desired levels. How much memory have you got left in each of the nodes while they’re doing tasks. The more nodes you have in your rack, the more information you will have to go through. Luckily there is software out there that can help you visualise such information all at once in various ways.
In this tutorial you will learn:
- How to install the
glancesmonitoring tool - How to configure glances
- How to use
glancesto see cluster information - How to create a local webpage to display the
glancesoutput
Building Raspberry Pi Series:
- Building a Raspberry PI Cluster – Part I: Hardware Acquisition and Assembly
- Building a Raspberry PI Cluster – Part II: Operating system installation
- Building a Raspberry PI Cluster – Part III: Simultaneous Node Management
- Building a Raspberry PI Cluster – Part IV: Monitoring
Software Requirements and Conventions Used
| Category | Requirements, Conventions or Software Version Used |
|---|---|
| System | Raspbian Linux |
| Software | Glances |
| Other | Privileged access to your Linux system as root or via the sudo command. |
| Conventions |
# – requires given linux commands to be executed with root privileges either directly as a root user or by use of sudo command$ – requires given linux commands to be executed as a regular non-privileged user |
Glances – the must-have in cluster monitoring
Think of glances as a top for clusters. It’s more than that. It’s a Swiss army knife for monitoring clusters. It can display everything you need in one terminal window: hardware temperature, fan speed, running processes, RAM and CPU usage, node IP and several other interesting, smaller gems. The software is based on ncurses and it will also let you browse your cluster nodes easily, configure what the information window displays and it even has a webserver mode for you to output that information in a webpage.
Installing Glances
The software is based on Python. There are several ways to install it. Most Linux distributions already have it in their repositories so in Ubuntu you would simply do a
$ sudo apt install glances
and all the package dependencies would be resolved. If you would rather do it the Python way, you can use pip:
$ sudo apt install pip $ pip install glances
Glances has a ton of modules such as RAID information, WiFi network information, web interface or Docker integration. To be sure you get them all, should you use pip, install glances with
$ pip install 'glances[action,browser,cloud,cpuinfo,docker,export,folders,gpu,graph,ip,raid,snmp,web,wifi]'
You can also use a setup script that installs glances on your system regardless of what distribution you use. It downloads the latest master branch from GitHub and sets it up on your system. For this use
$ curl -L https://bit.ly/glances | /bin/bash
Also be sure to install glances on all the cluster nodes.
Configuring Glances
After installation, glances creates a configuration file in /etc/glances/ called glances.conf. In this file you can fine-tune what and how the ncurses interface will show you. For example, if you want to use a white background instead of a black background in the system information screen you can modify the line
curse_theme=black
to
curse_theme=white
The ncurses interface uses colors so if you want to be visually allerted by an overloading CPU you can define at what percent the CPU usage bar will turn from green to red. For this modify the value of cpu_critical=90 to something between 10 and 99. RAM, SWAP, Load thresholds all have the same parameters:
careful=50 warning=70 critical=90
You can modify their values so that their respective bars will change colors only then that limit defined by the value is reached. Uncomment the Sensors alias lines in the [sensors] section to get information about CPU and motherboard temperature. For this you will also need to have the hddtemp and psutil packages installed on each of the Raspbian cluster nodes:
$ sudo apt install hddtemp python-psutil
The list of configurable elements is extensive but the /etc/glances/glances.conf file is well documented and self-explanatory and would require an article of its own. But let us see what you can do with glances.
Using Glances
There are several ways you can use glances. If you want to see information about a particular cluster node and you happen to be connected to it via SSH you can simply invoke the command with
$ glances
If you want to start glances in server mode on a cluster node you append the -s attribute:
$ glances -s
Then on another machine you can start glances as a simple client application and connect to the glances server you just started with
$ glances -c 192.168.x.x
where 192.168.x.x is the Glances server’s IP. Alternatively you can use hostnames if they are defined in your /etc/hosts file:
$ glances -c rpi1
You can define your cluster nodes in the configuration file. Look for the Client/server section in glances.conf and add your nodes:
[serverlist] # Define the static servers list server_1_name=192.168.1.126 server_1_alias=rpi1 server_1_port=61209 server_2_name=192.168.1.252 server_2_alias=rpi2 server_2_port=61209 server_3_name=192.168.1.150 server_3_alias=rpi3 server_3_port=61209 server_4_name=192.168.1.124 server_4_alias=rpi4 server_4_port=61209
As you can see, each server has a number, a name, an alias and a port. The port should remain 61209 for all nodes while the alias should correspond to the hostname of each cluster node IP.
When using multiple nodes, to avoid entering their login password each time you connect to a glances server you can define that password in plaintex in the glances configuration file. Edit /etc/glances/glances.conf and add your cluster nodes to the [passwords] section:
rpi1=yourpassword rpi2=yourpassword rpi3=yourpassword rpi4=yourpassword
Substitute yourpassword with the password you set in part three of this series and save the file.
Now you can call glances from a laptop with
$ glances --browser
and see all your cluster nodes in a browsable ncurses list. You can enter each of them, see what each one is doing with its resources and processes then move on to the next.
Webserver mode
But what if you wanted to view all of your cluster nodes at once? Would that be nicer? And wouldn’t it be even better to display that information in a webpage? Glances has a webserver mode that you can start with:
$ glances -w
Once a Glances server is activated as a webserver you can use any web browser to access a webpage detailing the same information you can view in the ncurses interface. Just input the node’s IP address or hostname in the URL bar of your browser and use 61208 as a port.
For example http://192.168.1.252:61208 will, in our LAN, display the Glances webserver’s information pertaining the second node of the cluster. The problem is you will have to use different pages and addresses for each node. So why not make a single webpage that displays all the four nodes at once using a simple HTML page with IFRAMES?
Creating a Glances webpage monitoring station
Install Apache on the first node of the cluster. We will use rpi1 as the node that displays this HTML file. You can do this with
$ sudo apt install apache2
The root of the Apache webserver is the /var/www/html/ folder. Create a new file here and name it index.html:
# touch /var/www/html/index.html
Now edit this file:
# nano /var/www/html/index.html
Add this to its contents:
out.println("<!DOCTYPE html>");
out.println("<html>");
out.println("<body>");
out.println("<h2>My cluster info"</h2>");
out.println("<br>");
out.println("<center>");
out.println("<iframe src="http://rpi1:61208" height="465" width="326"></iframe>");
out.println("<iframe src="http://rpi2:61208" height="465" width="326"></iframe>");
out.println("<iframe src="http://rpi3:61208" height="465" width="326"></iframe>");
out.println("<iframe src="http://rpi4:61208" height="465" width="326"></iframe></center>");
out.println("</br>");
out.println("</body>");
out.println("</html>");Save the file with ctrl+x. As you can see we are using hostnames instead of IPs and the Glances webserver port – 61208. There are four HTML IFRAMES, one for each cluster node. Change the height and width values so that the iframes will fill your screen.
Now just one more thing to do. Add the following line to each /etc/rc.local file of your cluster nodes:
glances -w --theme-white --process-short-name --hide-kernel-threads --fs-free-space --disable-irq --disable-raid --disable-folder --disable-swap
This tells Glances to start at boot as a webserver, to use the white theme, trim the process list width it displays and hide a few things that a Raspberry Pi does not need or that you would rarely use: RAID information, IRQ information, certain folder information, SWAP and kernel thread information. This is so that after you reboot your cluster and access the first node’s address through a web browser, the IFRAMES would be able to fit all relevant information in their respective windows.
You can further decide how the Glances webserver webpages display information by editing their respective CSS file located at /usr/lib/python3/dist-packages/glances/outputs/static/public/css/style.css. Font size, font type, colors – all can be customized here to make the cluster information webpage prettier.
Conclusion
Glances is a great way to get relevant information out of your cluster. If you use the IFRAME webpage method you can visit the cluster information page at any time and see how all the nodes are performing in one browser window. Or you can use the ncurses interface in a Terminal screen to view the same information. Glances is highly customizable and highly useful when you are using a cluster, be it a small one made out of Raspberry Pi’s.
Appendix
There are no limits. In case the 4 node Raspberry PI cluster is not up your satisfaction you can always add more nodes any time.
