monitors:bb-zfs

bb-zfs

Author Vernon Everett
Compatibility Xymon 4.2
Requirements ZFS
Download None
Last Update 2020-04-20

An update to the BB ZFS monitor to graph capacity and keep tabs on snapshots as well

Client side

  • Copy bb-zfs.ksh to the hobbit/xymon ext directory (usually in HOBBITCLIENTHOME/ext)
  • Add the following to clientlaunch.cfg
        [zfs]
            ENVFILE $HOBBITCLIENTHOME/etc/hobbitclient.cfg
            CMD $HOBBITCLIENTHOME/ext/bb-zfs.ksh
            LOGFILE $HOBBITCLIENTHOME/logs/zfs.log
            INTERVAL 5m

Server side

  • Add the following lines to hobbitgraph.cfg
    [zfs]
        FNPATTERN ^zfs,(.*).rrd
        TITLE ZFS Capacity
        YAXIS % Full
        DEF:p@RRDIDX@=@RRDFN@:lambda:AVERAGE
        LINE2:p@RRDIDX@#@COLOR@:@RRDPARAM@
        -u 100
        -l 0
        GPRINT:p@RRDIDX@:LAST: \: %5.1lf (cur)
        GPRINT:p@RRDIDX@:MAX: \: %5.1lf (max)
        GPRINT:p@RRDIDX@:MIN: \: %5.1lf (min)
        GPRINT:p@RRDIDX@:AVERAGE: \: %5.1lf (avg)\n
  • In hobbitserver.cfg, add the following line
 GRAPHS="...,xymond,zfs"
TEST2RRD="...,xymond,zfs=ncv"
SPLITNCV_zfs="*:GAUGE"

Show Code ⇲

Hide Code ⇱

#!/bin/ksh
# Revision History:
# 1. Mike Rowell <Mike.Rowell@Rightmove.co.uk>, original
# 2. Uwe Kirbach <U.Kirbach@EnBW.com>
# 3. T.J. Yang: add in some comments.
# 4. Vernon Everett <everett.vernon@gmail.com : Added check for old snapshots
#                                             : Added graphing data
# 5. FreeSoftwareServers <admin@freesoftwareservers.com>: 
# Update to new "zpool list -h" format
# Add Available/Free to table
# Full "zpool status" always displayed
# Set DEGRADED=RED
# DISKYELL=90 & DISKRED=95

DISKYELL=90
DISKRED=95
SNAPRED=90      # Days old
SNAPYELL=60     # Days old
SNAPCOL=true    # Set to true if snapshot age should effect test colour
CHECKSNAPS=true # Set to true to do snapshot checking
TEST="zfs"
DISPCOLOR="green"
FIRST_LINE="zfs - okay"
FIRST_LINE_HEALTH="okay"
FIRST_LINE_CAP="okay"
DATA=""

#What: beautify the page display by html code.
STRING="<table border=1 cellpadding=10><tr><th></th><th>Zpool Name</th><th>Status</th><th>Allocated</th><th>Free</th><th>Capacity</th></tr>"
#What: a loop to parse output of "zpool list -H" output.
# bash-3.00# zpool list
# NAME                    SIZE    USED   AVAIL    CAP  HEALTH     ALTROOT
# mypool                 33.8G   84.5K   33.7G     0%  ONLINE     -
#####UPDATE####
#
#:# zpool list
#NAME     SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
#raid-z  18.1T  13.6T  4.54T         -     0%    74%  1.00x  ONLINE  -
# bash-3.00# zpool list -H
# mypool  33.8G   84.5K   33.7G   0%      ONLINE  -
# bash-3.00#


zpool list -H | while read name size alloc free expandsz frag cap dedup health altroot
do
  LINE_COLOR="green"

  if [ "${health}" == "ONLINE" ]; then
    HEALTH_COLOR="green"
  elif [ "${health}" == "DEGRADED" ]; then
    HEALTH_COLOR="red"
  elif [ "${health}" == "FAULTED" ]; then
    HEALTH_COLOR="red"
  fi


  CAP_COLOR="green"
  cap=`echo ${cap} | cut -d% -f1`
  if [ "${cap}" -ge $DISKYELL ]; then
    CAP_COLOR="yellow"
  elif [ "${cap}" -gt $DISKRED ]; then
    CAP_COLOR="red"
  fi
  DATA=$(echo "$name : $cap\n$DATA")

# Determine the line colours
  [ "$HEALTH_COLOR" == "yellow" -o "$CAP_COLOR" == "yellow" ] && LINE_COLOR="yellow"
  [ "$HEALTH_COLOR" == "red" -o "$CAP_COLOR" == "red" ] && LINE_COLOR="red"

# Determine the messages
  [ "$HEALTH_COLOR" == "red" -a "$FIRST_LINE_HEALTH" != "faulted" ] && FIRST_LINE_HEALTH="DEGRADED"
  [ "$HEALTH_COLOR" == "red" -a "$FIRST_LINE_HEALTH" != "DEGRADED" ] && FIRST_LINE_HEALTH="FAULTED"
  [ "$CAP_COLOR" == "yellow" -a "$FIRST_LINE_CAP" != "full" ] && FIRST_LINE_CAP="nearly full"
  [ "$CAP_COLOR" = "yellow" ] && FIRST_LINE_CAP="full"

#Determine the final colour status
  [ "$LINE_COLOR" == "yellow" -a "$DISPCOLOR" != "red" ] && DISPCOLOR="yellow"
  [ "$LINE_COLOR" == "red" ] && DISPCOLOR="red"

STRING="$STRING <tr><td>&${LINE_COLOR}</td><td>${name}</td><td>${health}</td><td>${alloc}</td><td>${free}</td><td>${cap} %</td></tr>"
done
DATA=$(echo "$DATA \n\n")

STRING="$STRING </table><br><br>"
STRING="$STRING`zpool status`"
FIRST_LINE="zfs - Health Report: $FIRST_LINE_HEALTH - Capacity Report: $FIRST_LINE_CAP"

# Snapshot check
if [ "$CHECKSNAPS" = "true" ]
then
   NOW=$(perl -e 'print time(), "\n" ')
   SNAPLIST=$(zfs list | grep @ |awk '{ print $1 }' \
                       | while read a
                         do
                            echo "$(zfs get -H -o name,value -p creation $a) $( zfs get -H -o value used $a)"
                         done)
   if [ -z "$SNAPLIST" ]
   then
      SNAPTABLE="<br>&green No snapshots found"
   else
      SNAPTABLE="<table border=1 cellpadding=10><tr><th></th><th>Snapshot</th><th>Age</th><th>Size</th></tr>"
      echo "$SNAPLIST" | while read SNAPSHOT CREATION SIZE
      do
         LINE_COLOR=green
         ((SNAPREDS=SNAPRED*43200))
         ((SNAPYELLS=SNAPYELL*43200))
         ((AGES=NOW-CREATION)) # AGES=Age in seconds
         [ $AGES -gt $SNAPYELLS ] && LINE_COLOR=yellow
         [ $AGES -gt $SNAPREDS ] && LINE_COLOR=red
         [ $AGES -gt 120 ] && ((AGE=AGES/60))&& AGE="$AGE Minutes"
         [ $AGES -gt 7200 ] && ((AGE=AGES/3600)) && AGE="$AGE Hours"
         [ $AGES -gt 172800 ] && ((AGE=AGES/86400)) && AGE="$AGE Days"
         SNAPTABLE="$SNAPTABLE <tr><td>&${LINE_COLOR}</td><td>${SNAPSHOT}</td><td>${AGE}</td><td>${SIZE}</td></tr>"
         if [ "$SNAPCOL" = "true" ] #Only if true will it effect test colour
         then
            if [ "$DISPCOLOR" != "red" ] # If it's already red, it's not getting any worse
            then
               [ "$LINE_COLOR" != "green" ] && DISPCOLOR=$LINE_COLOR
            fi
         fi
      done
      SNAPTABLE="$SNAPTABLE </table><br><br>"
   fi
   STRING="$STRING <br><br><B>SNAPSHOT STATUS</B> $SNAPTABLE"
fi
# What: Sent out the final bb message to hobbit server.
$BB $BBDISP "status $MACHINE.$TEST $DISPCOLOR `date` $FIRST_LINE $STRING"
$BB $BBDISP "data $MACHINE.$TEST $DISPCOLOR $(echo; echo "$DATA" ;echo;echo)"

None known at this stage.

Go for lunch.

I am eternally grateful to those that have assisted in creating this script. All I have done is add a little extra functionality to monitor snapshots and a little bit of graphing.

From the script, it appears credit is owed to

  • 2010-06-10
    • Put it on Xymonton with snapshot feature
  • 2010-07-21
    • Now with graphing goodness
  • 2011-05-14
    • Fixed a minor code bug pointed out by Patrik Nilsson (Damn Quotes)
      It appears there is a difference between THIS=“THAT” and
       THIS="THAT" 

      At least if you cut and paste from this page there is. :-(

  • 2013-07-16
    • Changed graphing FNPATTERN to ^zfs,(.*).rrd
      Without the ^ it will pick up and try and graph any rrd file with the string “zfs,” in the file name.
  • 2020-04-20
    • Update to new “zpool list -h” format
    • Add Available/Free to table
    • Full “zpool status” always displayed
    • Set DEGRADED=RED
    • DISKYELL=90 & DISKRED=95
  • monitors/bb-zfs.txt
  • Last modified: 2020/04/21 01:11
  • by freesoftwareservers