Table of Contents

jmxstat

Author Andy Smith
Compatibility Xymon 4.2,4.3
Requirements Java 1.5 or above and jmxsh (http://code.google.com/p/jmxsh/)
Download jmxstat_kit.zip
Last Update 2017-11-17

Description

This is a tcl script that runs under jmxsh to connect to a JMX enabled JVM, extract stats and send to Xymon. It currently monitors Tomcat on a variety of UNIX, Linux and Windows servers. It is modeled on the beastat.pl monitor for Weblogic (http://sourceforge.net/projects/hobbit-perl-cl) but jmxstat should work for any JVM that exposes the JMX interface, especially Tomcat, but also JBoss and Weblogic. The jmxstat monitor provides the same reporting and alerting facilities as beastat and is configured in a very similar way so anyone familiar with beastat should be comfortable with the installation.

jmxstat is expected to run as a 'traditional' extension and makes use of a communications program provided with the local Xymon agent such as xymon, xymon.ps1 or BBWinCMD.exe to deliver the updates to the central Xymon server. See the README in the zipfile for discussion of the implementation differences between UNIX/Linux and Windows agents.

Installation

Client side

1. Copy the 3 files jmxsh-R5.jar, jmxstat.tcl and jmxstat.sh to the Xymon extensions directory $XYMONHOME/ext

2. Rename jmxstat_sample.ini to jmxstat.ini and copy it to the Xymon config directory $XYMONHOME/etc

3. Edit $XYMONHOME/etc/jmxstat.ini, as a minimum you will need to modify the following in the default section :- 

  adminport
  username
  password

4. Add entries to the hosts.cfg file (create one if none exists in your Xymon installation) [NOT required on Windows Clients] 

  0.0.0.0 some_unique_name_in_xymon_space # noconn jmxstat

5. Edit the $XYMONHOME/etc/clientlaunch.cfg and insert the following text: [Windows Clients handle this differently - see README] 

 [jmxstat]
      #DISABLED
      ENVFILE /apps/xymon/client/etc/xymonclient.cfg
      CMD $XYMONHOME/ext/jmxstat.sh
      LOGFILE $XYMONCLIENTLOGS/jmxstat.log
      INTERVAL 5m

Server side

6. Add this to TEST2RRD= in xymonserver.cfg 

Threads=ncv,JVMHeap=ncv,Classes=ncv,WebApp=ncv,ThreadPool=ncv,Service=ncv,GCInfo,jmxstat

7. Add this to GRAPHS= in xymonserver.cfg 

Threads,Classes,Service,JVMHeap,GCInfo,WebApp,ThreadPool,jmxstat

8. Add these 7 extra lines to xymonserver.cfg 

NCV_Threads="LiveThreads:GAUGE,Peak:GAUGE,DaemonThreads:GAUGE,Total:DERIVE"
NCV_JVMHeap="HeapSize:GAUGE,Committed:GAUGE,MaxHeapSize:GAUGE"
NCV_Classes="Classes:GAUGE"
NCV_WebApp="activeSessions:GAUGE,sessionCounter:GAUGE,maxActive:GAUGE,sessionMaxAliveTime:GAUGE,sessionAverageAlive:GAUGE,rejectedSessions:GAUGE,expiredSessions:GAUGE"
NCV_ThreadPool="currentThreadCount:GAUGE,currentThreadsBusy:GAUGE"
NCV_Service="bytesReceived:DERIVE,bytesSent:DERIVE,errorCount:DERIVE,requestCount:DERIVE"
NCV_jmxstat="runtime:GAUGE"

9. Add these 8 new sections to graphs.cfg 

[Threads]
      # java.lang:type=Threading
      TITLE Java Resources
      YAXIS Threads
      DEF:Lve=Threads.rrd:LiveThreads:AVERAGE
      DEF:Pek=Threads.rrd:Peak:AVERAGE
      DEF:r=Threads.rrd:Total:AVERAGE
      CDEF:Tot=r,60,*
      LINE2:Lve#00CCCC:Live Threads
      LINE2:Tot#FF0000:New Threads
      COMMENT:\n
      GPRINT:Lve:LAST:Current Threads \: %5.1lf%s (cur)
      GPRINT:Lve:MAX: \: %5.1lf%s (max)
      GPRINT:Lve:MIN: \: %5.1lf%s (min)
      GPRINT:Lve:AVERAGE: \: %5.1lf%s (avg)\n
      GPRINT:Tot:LAST:New Threads/min \: %5.1lf%s (cur)
      GPRINT:Tot:MAX: \: %5.1lf%s (max)\n    
[Classes]
      #java.lang:type=ClassLoading
      TITLE Java Resources
      YAXIS Classes
      DEF:count=Classes.rrd:Classes:AVERAGE
      LINE2:count#00CCCC:Classes Loaded
      GPRINT:count:LAST: %6.1lf (cur) \:
      GPRINT:count:MAX: %6.1lf (max) \:
      GPRINT:count:MIN: %6.1lf (min) \:
      GPRINT:count:AVERAGE: %6.1lf (avg)\n
[Service]
      # Catalina:type=GlobalRequestProcessor,name=http-8080
      TITLE Java Resources
      YAXIS Requests/sec
      DEF:in=Service.rrd:bytesReceived:AVERAGE
      DEF:out=Service.rrd:bytesSent:AVERAGE
      DEF:r=Service.rrd:requestCount:AVERAGE
      CDEF:req=r,1,*
      -X0
      LINE2:req#FF0000:Requests
      COMMENT:\n
      GPRINT:req:LAST:Requests/s      \: %5.1lf%s (cur)
      GPRINT:req:MAX: \: %5.1lf%s (max)
      GPRINT:req:AVERAGE: \: %5.1lf%s (avg)\n
      GPRINT:in:LAST:Received bytes/s\: %5.1lf%s (cur)
      GPRINT:in:MAX: \: %5.1lf%s (max)
      GPRINT:in:AVERAGE: \: %5.1lf%s (avg)\n
      GPRINT:out:LAST:Sent     bytes/s\: %5.1lf%s (cur)
      GPRINT:out:MAX: \: %5.1lf%s (max)
      GPRINT:out:AVERAGE: \: %5.1lf%s (avg)\n
[JVMHeap]
      TITLE JVM Heap Memory
      YAXIS Memory
      DEF:kused=JVMHeap.rrd:HeapSize:AVERAGE
      DEF:ktotal=JVMHeap.rrd:Committed:AVERAGE
      CDEF:bused=kused,1024,*
      CDEF:btotal=ktotal,1024,*
      CDEF:free=btotal,bused,-
      CDEF:used=bused
      -l 0
      -b 1024
      AREA:used#0000FF:Used Memory
      AREA:free#00FF00:Free Memory:STACK
      COMMENT:\n
      GPRINT:used:LAST:Used \: %8.1lf%s (cur)
      GPRINT:used:MAX: \: %8.1lf%s (max)
      GPRINT:used:MIN: \: %8.1lf%s (min)
      GPRINT:used:AVERAGE: \: %8.1lf%s (avg)\n
      GPRINT:free:LAST:Free \: %8.1lf%s (cur)
      GPRINT:free:MAX: \: %8.1lf%s (max)
      GPRINT:free:MIN: \: %8.1lf%s (min)
      GPRINT:free:AVERAGE: \: %8.1lf%s (avg)\n
[GCInfo]
      TITLE Garbage Collection Activity
      YAXIS # collections
      FNPATTERN GCInfo.(.+).rrd
      DEF:r@RRDIDX@=@RRDFN@:CollectionCount:AVERAGE:step=300
      DEF:t@RRDIDX@=@RRDFN@:CollectionTime:AVERAGE:step=300
      CDEF:count@RRDIDX@=r@RRDIDX@,600,*
      CDEF:duration@RRDIDX@=t@RRDIDX@,600,*,1000,/,count@RRDIDX@,/
      LINE1:count@RRDIDX@#@COLOR@:@RRDPARAM@ occurances\:
      -X0
      GPRINT:count@RRDIDX@:MAX: %6.0lf (max)
      GPRINT:count@RRDIDX@:AVERAGE:\: %6.0lf (avg)
      GPRINT:count@RRDIDX@:LAST:\: %6.0lf (last)\n
      LINE2:duration@RRDIDX@#@COLOR@:@RRDPARAM@   duration\:
      GPRINT:duration@RRDIDX@:MAX: %6.3lf (max)
      GPRINT:duration@RRDIDX@:AVERAGE:\: %6.3lf (avg)
      GPRINT:duration@RRDIDX@:LAST:\: %6.3lf (last) sec/run\n
      COMMENT: \n
[WebApp]
      # Catalina:type=Manager,path=/arsys,host=localhost
      TITLE WebApp Usage
      YAXIS Sessions
      DEF:cur=WebApp.rrd:activeSessions:AVERAGE
      DEF:r=WebApp.rrd:sessionCounter:AVERAGE
      CDEF:new=r,60,*
      LINE2:cur#00FF00:Current Sessions
      COMMENT:\n
      GPRINT:cur:LAST:Sessions        \: %5.1lf%s (cur)
      GPRINT:cur:MAX: \: %5.1lf%s (max)
      GPRINT:cur:MIN: \: %5.1lf%s (min)
      GPRINT:cur:AVERAGE: \: %5.1lf%s (avg)\n
      GPRINT:new:LAST:New Sessions/min\: %5.1lf%s (cur)
      GPRINT:new:MAX: \: %5.1lf%s (max)
      GPRINT:new:AVERAGE: \: %5.1lf%s (avg)\n
[ThreadPool]
      #Catalina:type=ThreadPool,name=http-8080
      TITLE WebApp Usage
      YAXIS Thread Pool
      DEF:count=ThreadPool.rrd:currentThreadCount:AVERAGE
      DEF:active=ThreadPool.rrd:currentThreadsBusy:AVERAGE
      LINE2:count#00CCCC:Current Threads
      LINE2:active#FF0000:Busy Threads
      COMMENT:\n
      GPRINT:count:LAST:Current\: %5.1lf%s (cur)
      GPRINT:count:MAX: \: %5.1lf%s (max)
      GPRINT:count:AVERAGE: \: %5.1lf%s (avg)\n
      GPRINT:active:LAST:Busy   \: %5.1lf%s (cur)
      GPRINT:active:MAX: \: %5.1lf%s (max)
      GPRINT:active:AVERAGE: \: %5.1lf%s (avg)\n
[jmxstat]
      TITLE jmxstat Run Time
      YAXIS Seconds
      DEF:rt=jmxstat.rrd:runtime:AVERAGE
      AREA:rt#00CCCC:Run Time
      COMMENT:\n
      GPRINT:rt:LAST: \: %5.1lf (cur)
      GPRINT:rt:MAX: \: %5.1lf (max)
      GPRINT:rt:MIN: \: %5.1lf (min)
      GPRINT:rt:AVERAGE: \: %5.1lf (avg)\n

10. This step is not needed if you have the zonestat or mpstat monitor installed. 

# "rrddata" updates RRD files with information that arrives as "data" messages.
# If you want RRD graphs of your monitoring BB data, then you want to run this.

[rrddata]
    ENVFILE /home/xymon/server/etc/xymonserver.cfg
    NEEDS xymond
    CMD xymond_channel --channel=data  
           --log=$XYMONSERVERLOGS/rrd-data.log xymond_rrd
           --extra-tests=mpstat,zonestat,GCInfo
           --extra-script=/home/xymon/server/ext/rrd_data.pl
           --rrddir=$XYMONVAR/rrd

11. Add the following to your copy of rrd_data.pl 

} elsif ( $testname eq "GCInfo" ) {
##
## Reads in the jmxstat.tcl from client
##
   if ( defined($buffer{"GarbageCollectors"}) ) {
      @loop=split(' ',$buffer{"GarbageCollectors"});

      print "DS:CollectionCount:DERIVE:600:0:U\n";
      print "DS:CollectionTime:DERIVE:600:0:U\n";

      foreach $key ( @loop ) {
         undef(@line);
         push(@line,$buffer{"${key}CollectionCount"});
         push(@line,$buffer{"${key}CollectionTime"});

         $tmp = (split('/',$key))[1];
         print "$testname.${key}.rrd\n";
         $line=join(':',@line);
         print "$line\n";
      }

   }
}

Source

jmxstat.tcl

The tcl code on its own is quite substantial (2000 lines or so) but in order to be somewhat independent of the local environment, I have included code from tclib, so the whole script is 6000 lines or so, too much to paste onto this page. I have created an archive of the various materials which you can fetch from the download URL above.

Show Code ⇲

Hide Code ⇱ 








Known  Bugs and Issues









To Do









Credits









Changelog