Howto: graph network traffic using RRDtool

[gonk]

My objective was to visualize the network usage on my B3.

Planning and preparations

Install RRDtool
If you do not have RRDtool installed:

Code: Select all

apt-get install rrdtool

For all the info you need on RRDtool please visit http://oss.oetiker.ch/rrdtool/ or google it.

Decide what to monitor
The network interfaces on your B3 are listed by using the /sbin/ifconfig command, run it as root and pick the interfaces you'd like to monitor.

Code: Select all

su
/sbin/ifconfig

I selected to monitor wlan0, br0, eth0, eth1 and lo.

Decide locations
You need to decide where you want to keep your scripts, your RRD data etc.

In my case I have the scripts in bin/ in my user folder and the data in var/rrddata, also in my user folder. You may have a different opinion and thus need to change some parts accordingly.

Decide your RRD preferences
When you familiarized yourself with RRDtool you learned that the database(s) are set up to keep different resolutions of the data over different amounts of time before aggregating.

The examples below are for taking a measurement every minute and then holding

• 7 days with 1 min resolution (10080 data points, 1 steps per point)
• 60 days with 5 min resolution (17280 / 5)
• 1 year with 10 min resolution (52560 / 10)
• 5 years with 1h resolution (43800 / 60 steps)

Execution

Once you've prepared your setup you should be avle to follow these steps.

Create databases
Run the following script (i called it create_rrd_network-interfaces.sh) for each interface you'd like to monitor, e.g.

Code: Select all

create_rrd_network-interfaces.sh eth0

Code: Select all

#!/bin/bash
# Script to create rrd-file

#  7d with 1 min resolution 	=10080 /  1 step
# 60d with 5 min resolution	=17280 /  5 steps
#  1y with 10 min resolution	=52560 / 10 steps 
#  5y with 1h resolution	=43800 / 60 steps

# add "--start timestamp" before --step, if needed

if [ "$1" = "" ] ; then
  echo "parameter: name-of-interface (e.g. br0, wlan1)"
else
   directory="/home/[b]my-user-name[/b]/var/rrddata/"
   filename="network-interface-$1.rrd"

   # Check i file already exists
   if [ ! -f "$directory$filename" ]
   then
	# File doesn't exist, create new rrd-file
	echo "Creating RRDtool DB"
	rrdtool create $directory$filename \
		 --step 60 \
		 DS:RXbytes:DERIVE:120:0:U \
		 DS:RXpackets:DERIVE:120:0:U \
		 DS:TXbytes:DERIVE:120:0:U \
		 DS:TXpackets:DERIVE:120:0:U \
		 RRA:AVERAGE:0.5:1:10080 \
		 RRA:AVERAGE:0.5:5:17280 \
		 RRA:AVERAGE:0.5:10:52560 \
		 RRA:AVERAGE:0.5:60:43800 \
		 RRA:MAX:0.5:1:10080 \
		 RRA:MAX:0.5:5:17280 \
		 RRA:MAX:0.5:10:52560 \
		 RRA:MAX:0.5:60:43800 \
		 RRA:MIN:0.5:1:10080 \
		 RRA:MIN:0.5:5:17280 \
		 RRA:MIN:0.5:10:52560 \
		 RRA:MIN:0.5:60:43800
	echo "Done!"
   else
	echo $directory$filename" already exists, delete it first."
   fi
fi

Change my-user-name and path to whatever is relevant to you.

Create scripts to gather data and insert into the RRD's
I created one script (bin/rrd_update_network_interfaces_all.sh) to be run by cron that loops over the interfaces and calls another script (bin/rrd_update_network_interface.sh) to do the work.

bin/rrd_update_network_interfaces_all.sh

Code: Select all

#!/bin/bash

# Run as root, otherwise ifconfig is not available.
# NB! You need to specify the complete path to rrd_update_network_interface.sh, otherwise the script will not be found when executing as root.

time=`date +%s`

for interface in br0 eth0 eth1 lo wlan0
do
   echo updating RRD for $interface
   /home/[b]my-user-name[/b]/bin/rrd_update_network_interface.sh $interface $time
done

Change the listed interfaces in the for statement if your preferences differ from mine.

rrd_update_network_interface.sh

Code: Select all

#!/bin/bash

# run as root, otherwise ifconfig is not available

if [ $# -eq 2 ]
then
   interface=$1
   timestamp=$2
   directory="/home/[b]my-user-name[/b]/var/rrddata/"
   filename=$directory"network-interface-$interface.rrd"

   # Check if RRD file exists
   if [ -f "$filename" ]
   then
	TXbytes=`/sbin/ifconfig $interface | grep "RX bytes" | sed 's/ RX bytes//;s/ TX bytes//' | cut -d':' -f3 | cut -d' ' -f1`
	RXbytes=`/sbin/ifconfig $interface | grep "RX bytes" | sed 's/ RX bytes//;s/ TX bytes//' | cut -d':' -f2 | cut -d' ' -f1`
	TXpackets=`/sbin/ifconfig $interface | grep "TX packets" | cut -d':' -f2 | cut -d' ' -f1`
	RXpackets=`/sbin/ifconfig $interface | grep "RX packets" | cut -d':' -f2 | cut -d' ' -f1`

echo rrdtool update $filename $timestamp:$TXbytes:$RXbytes:$TXpackets:$RXpackets
	rrdtool update $filename $timestamp:$TXbytes:$RXbytes:$TXpackets:$RXpackets
   else
	echo $filename" doesn't exist."
   fi
else
   echo "parameter: name-of-interface (e.g. br0, wlan1) timestamp"
fi

Change my-user-name and path to whatever is relevant to you.

Set up cron job
I did this the dirty way but be wise.

Code: Select all

su
nano /etc/crontab

/etc/crontab

Code: Select all

#RRD
#----------
# m     h  dom mon dow  user    command
*/1     *  *   *   *    root    /home/ [b]my-user-name[/b]/bin/bin/rrd_update_network_interfaces_all.sh

Change my-user-name and path to whatever is relevant to you.

Creating graphs

This part I've not detailed yet but feel free to contribute.
Reference: http://oss.oetiker.ch/rrdtool/doc/rrdgraph.en.html

[nobody]

Wouldnt

Code: Select all

Apt-get install munin munin-node

give you the same result?

[gonk]

I tried Cacti and it was so good at making something simple very hard and with very poor documentation so I went for the bare bone implementation.

Is munin easy to configure? Cacti sucked badly and had no proper and structured documentation.

[ahoff]

to avoid that the mailbox get full, add > / dev / null 2> & 1 in crontab:

Code: Select all

#RRD
#----------
# m     h  dom mon dow  user    command
*/1     *  *   *   *    root    /home/ [b]my-user-name[/b]/bin/bin/rrd_update_network_interfaces_all.sh > / dev / null 2> & 1

[nobody]

Yes munin is very user friendly. Writing custom monitoring graphs is also easy.

[gonk]

Yes munin is very user friendly. Writing custom monitoring graphs is also easy.

Do you have some good examples?

[gonk]

to avoid that the mailbox get full, add > / dev / null 2> & 1 in crontab:

Code: Select all

#RRD
#----------
# m     h  dom mon dow  user    command
*/1     *  *   *   *    root    /home/ [b]my-user-name[/b]/bin/bin/rrd_update_network_interfaces_all.sh > / dev / null 2> & 1

Thanks.

I bet that those spaces in "> / dev / null" were not intentional but what is correct for "2> & 1" and what does that imply?

Further, what do you mean by a mailbox problem? I've seen others touching upon similar topics on the forum but pleae explain.

[ahoff]

http://forum.excito.net/viewtopic.php?f=9&t=4398

[gonk]

http://forum.excito.net/viewtopic.php?f=9&t=4398

Code: Select all

root@B3:/home/my-user# su
root@B3:/home/my-user# cd
root@B3:~# du Mail
du: cannot access `Mail': No such file or directory
root@B3:~# ls -l
total 0

Thus... no Mail/, no problems.